When I meet with people and talk about our work, I get asked a lot what technology we use to create interactive and dynamic data visualizations. At Interactive Things, we have a set of preferred libraries, applications and services that we use regularly in our work. We will select the most fitting tool for the job depending on the requirements of the project. Sometimes a really simple tool is all you need to create something meaningful. On other occasions, a more multifaceted repertoire is needed. But how does one choose the right thing to use? An endless list of every tool available, does not answer that question and a recommendation from a friend is oftentimes more valuable.

That’s why we have put together a selection of tools that we use the most and that we enjoy working with. We called it selection.datavisualization.ch. It includes libraries for plotting data on maps, frameworks for creating charts, graphs and diagrams and tools to simplify the handling of data. Even if you’re not into programming, you’ll find applications that can be used without writing one single line of code. We will keep this list as a living repository and add / remove things as technology develops. We hope this will help you find the best tool for your next job.

Let me answer the most likely questions right away: No, not everything find its’ way into this list, so you might not find your personal favorite. Yes, some of the libraries mentioned are brand spanking new and may not be ready for production while others already have successors, but they still provide a good service.

I just left the Stamen studio where I had a brief chat with founder and CEO Eric Rodenbeck. He enthusiastically showed me their newest work: maps.stamen.com. As part of the CityTracking project, the savvy folks at Stamen have put create three sets of map tiles to be used with OpenStreetMap data. These tiles are not your average map using custom colored vector paths. Instead Eric and his team tried to push the boundaries of what we consider feasible in mapping on the web.

Toner is a high-contrast black and white tile set that is optimized to focus on the actual shapes found in maps without being distracted by a plethora of colored lines and areas. Eric mentioned that working with simplified maps like these, allows the creator to focus on the actual information that is represented on multiple levels to find the right balance between no information and an overload thereof.

Terrain contains hill shading and natural vegetation colors as seen many great static maps from the past (like the ones in this excellent book; no affiliation in any way, I just love this book).

Watercolor is what got me really excited: The watercolor tiles were created by applying raster effect area washes and organic edges over a paper texture. What started as an exploration of how water color behaves eventually was automated and simulated by the awesome work of Zach Watson. He would actually describe the process rather as faking than simulating, but the outcome is what counts.

Along the demo of the maps, Stamen also provides simple instructions how to use them with you favorite mapping framework like ModestMaps, Leaflet or Google Maps. They are available under a Creative Commons Attribution (CC BY 3.0) license.

Shan Carter‘s latest interactive for The New York Times offers four ways to slice Obama’s 2013 federal budget proposal. The $3.7 trillion in spending can be analyzed by type of spending, changes to discretionary spending, department totals and of course all together. In all of the charts, the data is represented as bubbles. The size equals the proposed amount and the color indicates a cut or an increase in spending from 2012. Although each chart has an individual layout, the transitions between them is extremely smooth and helps the user to make sense of the underlying data.

View the interactive Version at NYTimes.com

Over the past years we have experienced an increased understanding for the importance of open data. We’ve seen governments, non-governmental organizations and even private and public companies open up their data repositories. Beside the open data champions inside these entities that pushed the idea through ignorance, bureaucracy and firewalls, there has also been a lot of work done to provide guidance, standards and tools. One important contributor of such things is the Open Knowledge Foundation. The globally-located and community-focused team that nurtures communities and builds tools to promote open knowledge, data and content. I think it’s time to show my appreciation by highlighting some of their initiatives.

CKAN is an open-source data portal application that makes it easy to publish, share and find data. It provides a powerful database for cataloging and storing datasets, with an intuitive web front-end and API. The core functionality can be flexibly extended with the features needed – from social integration and comments, to Google Analytics, to integrated data storage.

TheDataHub.org is a community-run catalog of useful sets of data on the Internet. Users can collect links here to data from around the web for themselves and others to use, or search for data that others have collected. Depending on the type of data and its conditions of use, the Data Hub may also be able to store a copy of the data or host it in a database, and provide some basic visualization tools.

GetTheData.org is a Q&A site where users can ask data related questions like “where to find data relating to a particular issue”, “what tools to use to explore a data set in a visual way”, or “how to cleanse data or get it into a format you can work with using third party visualization or analysis tools”.

DataPatterns.org is a collection of tips and tricks for data work. It’s a collection of opinions and evolving best practices. The purpose is not to present all available options and technologies but to pick one and follow it through. The site is a collaborative community effort: if you have some good hacks and would like to share them, you can contribute a patch to the Data Patterns repository.

OpenDataManual.org discusses legal, social and technical aspects of open data. The manual can be used by anyone but is especially designed for those seeking to open up data. It discusses the why, what and how of open data – why to go open, what open is, and the how to open data.

OpenDefinition.org sets out principles to define ‘openness’ in relation to content, data and services — that’s any kind of material or data from sonnets to statistics, genes to geodata. In addition this site hosts the Open Software Service Definition which defines openness in relation to online (software) services.

OpenDataCommons.org provides a set of legal tools for open data. The site hosts three licenses for open data: the Public Domain Dedication and License (PDDL), the Attribution License (ODC-By), and the Open Database License (ODC-ODbL). Additionally, they provide the ODC Attribution-Sharealike Community Norm.

As data visualization enthusiasts, the topic of open data is dear to our hearts. Its’ relevance to data visualization has been proven by a lot of successful projects in the past, like CrimeSpotting.org or WhereDoesMyMoneyGo.org. While I am far from being an expert in this area, I believe the following three principles summarized from the full Open Definition are important objectives for visualization practitioners like you and me.

First, having honest, consistent and accurate data available ensures that our work has the solid foundation it takes to tell compelling stories and answer relevant questions. Second, having this type of data in a convenient and modifiable form that is easy to maintain and to extend makes visualizations and applications that are driven by this data much more sustainable. Third, having the data provided under terms that permit use, reuse and redistribution for everyone without discrimination against fields of endeavor, persons or groups. For example, ‘non-commercial’ restrictions that would prevent ‘commercial’ use in client projects are not allowed.

With technical, social, and legal tools at our hands we can push open data visualization forward. That’s why I respect the work of the Open Knowledge Foundation and all the contributors to their efforts to this degree. OKF, I salute you!

Berlin based designer and programmer Christopher Warnow had a closer look at the interest graph between people reading the same books. He developed an application that loads recommendations for a given book on Amzon.com and converts visualizes the information as a network. Soon, milieus of interest emerge showing related topics and additional literature.

The application is written in Processing leveraging the power of the open source graph visualization library Gephi Toolkit. You can download the tool and read more information on Christopher’s website.

Another big event in 2012 has just been announced: Resonate New Media Festival . On March 16-17 20 speakers from Europe, Asia and North America will come present at the Dom Omladine in Belgrade, Serbia. To the day program of talks, workshops and panel discussions the organizers added an awesome night program with concerts, DJ sets and visual performances. Looks like something you should not miss!

Resonate is setting new standards in the arts industry by creating a new platform for networking, information, knowledge sharing and education. It will bring together distinguished, world class artists, with an opportunity of participating in a forward-looking debate on the position of technology in art and culture. It’s more than just a festival, expert seminar or exhibition of visual arts. It is broad enough to encompass areas ranging from software engineering to visual arts theory, but also to create a bridge between culturally separated segments of the artistic and intellectual scene through a comprehensive, multidisciplinary approach.

The list of confirmed participants includes Nicholas Felton, Josh Nimoy, Jer Thorp, Greg J. Smith, Regine Debatty, Champagne Valentine, Niklas Roy, Benjamin Gaulon, Martial Geoffre-Rouland, Karsten Schmidt, FIELD, LAb[au], Rafaël Rozendaal, United Visual Artists, Written Images, Jürg Lehni, WARP, onedotzero and others.

We are happy and proud to support the Resonate 2012 Festival as a media partner and hope you all enjoy this exceptional event.

We are currently working with an intern here at Interactive Things: Flavio Gortana joined us three months ago and continues to surprise us with how fast he picks things up and how persistently he works on his ideas. After having worked on illustrations, iconography and interfaces it was time for him to get started with his first visualization project. “Designing Data Visualizations” should build the foundation for the work ahead. Here’s how Flavio experienced his first introduction to visualization, what he learned and how this helps him to move forward.

What the book taught me

Before starting something new, for me it is always important to know what I’m dealing with and to have a general knowledge about it. I’m not the kind of person able to just jump into something without thinking about it before. Since the only visualizations of real data I’ve made so far were some graphs in physics class, this book provided me with a basic vocabulary of terms and concepts needed to dive deeper into the topic.

I have been aware of my interest in visualized data for a long time and of course, I’ve already seen many different types of graphics that intended to explain a dataset. Until not so long ago, I have not given much thought to the quality, techniques or the concepts behind those. I just took them as they were and I tried to discern the information I was interested in. Of course, some even looked nice and had beautiful color schemes while others didn’t. The major benefit this book gave me was a fundamental change of how I look at a data visualization. It did so not by telling me how to analyze or criticize such graphics, but it provided a solid overview of visualization types and ways to distinguish between them. Furthermore, I learned about the underlying techniques and how to classify visualizations.

A good thing about Designing Data Visualizations — especially for a novice like me — is that it is rather practically oriented than theoretically. The explained concepts are very straight forward and I assume they are directly applicable to real problems. They are not some highly sophisticated theories without much practical relevance. Contrary to the “experimenting – mentality” of many design educations, the authors give clear advice what to do and what to avoid. They also state in the beginning that only one who knows the rules can break them in a suitable way. Of course, this may feel very constraining in the first place. But at the same time the book reveals the diversity of possibilities inside the given constraints. For me this will be helpful to get started very quickly. By exposing and explaining the different elements of data visualization, it also made me think of very trivial things I was not aware of before: Have you ever thought about color not having a natural order like saturation or the size of a shape? Or, have you ever thought about using the area instead of the radius while visualizing data with circles? I did not.

In conclusion I can say that the book answered many questions I had in the first place. It may not provide new or revolutionary concepts, but it shows and explains the existing ones in an understandable, manageable and accessible way. Although this won’t directly lead me to some groundbreaking visualizations, thanks to the good summary I will be aware of the variety of things to consider when visualizing data.

Students from the greater Berlin area gathered together on Saturday morning around 10am prepared to design and code away for the next 24 hours. The team behind Visualizing.org didn’t leave any wishes open and prepared excellent working conditions at the selected event location Urania. After a brief welcome message from GE the students learned about the data set they will try to make sense of. The data consisted of German demographics and health care statistics. The teams were assigned with the creation of a visualization that reveals true insights from the data and communicates them in an accessible, innovative and elegant way.

The Presentations

Before the student started working, they had the chance to listen to two of Germany’s best visualizers Moritz Stefaner and Gregor Aisch. With their presentations, they shed some light on the craft, process and peril from their daily work.

Moritz had prepared a packed deck of things that would have been helpful to know beforehand. It truly was a collection of golden tipps for newcomers and also some useful reminders for more experienced practitioners. He talked about how position is on of the strongest encoding techniques, what the difficulties are when working with colors and why we should not neclect uderused attributes like texture or hatching. As always he inspired many thoughts and provoked even more laughs with his content and delivery.

Gregor followed with the presentation of his daily routine as a freelance information visualizer. He introduced a process framework inspired by the highly recommended book Visualizing Data by William Cleveland. His iterative process consists of the following three phases:

1. Visualize the data using tools like R, Tableau or any other spreadsheet software to create sketches of what’s important, interesting or hidden in the data.
2. Learn from your results by noting down all your insights from the previous phase and explain your thoughts with the according charts. The duty of articulating your findings helps you to hone the vision for the ultimate/final solution.
3. Transform the data to achieve a more precise and more accessible visualization and therefore to draw clearer picture on the core message.

The Works

Soon after the presentations, the venue was filled with sketches on paper and whiteboards, laptops running calculations and enthusiastic people bouncing ideas off each other. The creative working athmosphere was truly stimulating and so Moritz, Gregor and myself couldn’t resist to start discussing and hacking as well. After an intense night, the teams handed in 11 visualizations that ranged from static information graphics to interactive visualizations. As I am involved in judging the works, I can’t comment on the works just yet, but Moritz sums it up nicely when he tweets: “I am really impressed by some of the works from the Visualizing Marathon Berlin 2011 #goodjob #toughtimesforjudges“. I recommend you have a look and judge for yourself here.

The Berlin chapter closes the 2011 series of the Visualizing Marathons. Charlene Manuel, of Visualizing.org had nothing but admiring words for the students works, efforts and enthusiasm. I assume nothing stands in the way of Visualizing Marathon 2012.

I am super hyped to introduce our pet project of this summer, called Substratum. It’s a series of inspirational interviews with some of today’s most influential designers and artists. 10 questions per interview, 2 interviews per issue, 1 issue per week until Christmas!

We start today with the first issue titled “Space” containing interviews by Marius Watz and Theo Watson & Emily Gobeille. A total of 14 interviews are in store for you and will be released on a weekly basis. The complete list of names featured in Substratum remains secret, but includes people like Ben Fry, Nicholas Felton and Golan Levin. Here is why we do this:

We want to feel the pulse of the creative community working at the intersection of art, design, and science. They are restlessly weaving together the physical with the digital, turning data into meaning, and creating interactive experiences to generate inspiring insights that encourage us to challenge our own works.

A lot has been talked about the works of these creative makers; we want to find out what they feel and think about their worlds. We want to get to know their beliefs, motivations and limitations. We want to know where they’re coming from and why they do what they do.

Move over to substratumseries.com and subscribe via RSS or Twitter to get updates of new released issues. We hope you enjoy what you see and read as much as we did putting everything together. If you have any questions, suggestions and thoughts for us, please let us know.

Jer Thorp, Data Artist in Residence at the New York Times and creative coder extraordinaire, explains the process behind his latest piece for Popular Science in a recent article. The task at hand was to visually represent the complete archive of their publication. The final piece is anchored by a kind of molecular chain – decade clusters in turn contain year clusters. Every atom in these year clusters is a single issue of the magazine, and is shaded with colours extracted from the issue covers via a colour clustering routine. The size of the issue-atoms is determined by the number of words in each issue.

Picking out interesting words from all of the available choices (pretty much the entire dictionary) was a tricky part of the process. I built a custom tool in Processing that pre-visualized the frequency plots of each word so that I could go through many, many possibilities and identify the ones that would be interesting to include in the final graphic. This is a really common approach for me to take – building small tools during the process of a project that help me solve specific problems. For this visualization, I actually ended up writing 4 tools in Processing – only one of which contributed visually to the final result.

It’s excellent to get a glimpse at how such complex yet elegant solutions come together and what decisions were taken along the way. See the related set on Flickr to visually follow the development of the graphic. This isn’t Jer’s first process documentation and if you are not familiar with his work, I strongly recommend to have a look.