You probably recognize that there are no objective measures to creating those "Top 10" and "Top 100" lists. There is a generous sprinkling of personal bias and subjective decisions.

Yet the assumption is that rankings of median home prices and crime rates and the "best places to live" aren't being done deceptively. Still, a way to account for unintentional bias would be great, and so Harvard researchers have created LineUp, an open-source application that empowers ordinary citizens to make quick, easy judgments about rankings based on multiple attributes.

In what they are billing as the first dynamic visualization software of its kind, LineUp allows users to assign weights to different parameters to create a custom ranking. For example, users might look at the raw data behind university rankings and decide for themselves the relative importance of student-faculty ratios or the number of citations per faculty member.

"It liberates people," says Alexander Lex, a postdoctoral researcher at the Harvard School of Engineering and Applied Sciences (SEAS). "Imagine if a magazine published a ranking of 'best restaurants.' With this tool, we don't have to rely on the editors' skewed or specific perceptions. Everybody on the Internet can go there and see what's really in the data and what part is personal opinion."

LineUp is part of a larger software package called Caleydo, an open-source visualization framework developed at Harvard, Johannes Kepler University, and Graz University of Technology. Caleydo visualizes genetic data and biological pathways—for example, to analyze and characterize cancer subtypes.

"LineUp really was developed to address our need to understand the ranking of genes by mutation frequency and other clinical parameters in a group of patients," explains  Hanspeter Pfister, An Wang Professor of Computer Science at SEAS. "It is an ideal tool to create and visualize complex combined scores of bioinformatics algorithms."

"We started thinking about how we can make this easy for biologists to understand and how we can tell them what the most important parts of the dataset are," says Lex.

While LineUp is still being applied to formal genetic research, the group has chosen to also apply their work to simpler, more familiar ranking problems—for example, the healthiness of different foods, best employers, or the best places to live.

LineUp introduces a dynamic element to the static analysis usually done on an Excel spreadsheet. It allows the user to immediately consider or ignore columns in a dataset by simply dragging them into or out of the window. It also enables side-by-side comparisons of alternative weighting systems.

And of course, not all metrics contribute to an item's rank the same way. Higher values of some metrics imply a higher rank, but not in all cases. LineUp has the ability to easily transform a dataset by inverting it, for example, making a lower crime rate correspond to a higher quality-of-life rank. Users can quickly apply and visualize the results of their intuitions.

Lex also emphasizes the potential for LineUp to be used predictively. Because LineUp makes it easy to pose many "what if" scenarios, a car manufacturer could devise a way to efficiently raise a vehicle's rank in a list of "best cars," perhaps by prioritizing fuel efficiency over style in the design process.

Previously, people without deep statistical or technical knowledge had no way of knowing authors' or institutions' potential biases or agendas. LineUp brings the public one step closer to simpler, personalized, and more meaningful data analysis.

"Essentially, it's a tool to allow people to explore the complexity of reality," says Lex.

The collaborators earned the best paper award at the IEEE Information Visualization (InfoVis) conference in October 2013 and the team is currently working on a web-based version that will make it easier to share and create rankings.