Christopher Manning

I wanted to build a project using a government dataset to learn d3.js. I decided to use the Chicago lobbyist data after looking at the information available at ChicagoLobbyists.org.

Now, to help you understand how to create your first visualization, I’ll share how I created this project.

Picking the Chart Type

There are a lot of excellent examples of the visualizations you can create using d3.js. Overall I wanted to display the relations between lobbies, clients, and agencies and the money associated with them. After evaluating all the available examples, I decided to use a force layout for this project. Next, I learned about the JSON data required for the layout after reading all the really well written documentation.

Generating the JSON Data

I used a local copy of the ChicagoLobbyists.org Sinatra app to generate the JSON need to create the visualization. Of course, the following is not very efficient with its N+1 queries, but I wanted the quickest and most straight forward solution since I wasn’t very familiar with DataMapper and would only need to extract the data once.

Learning and Using d3.js

D3.js has really good documentation. This really helped me get started with the library and get accustomed to the way you should use it to build your visualizations. I was able to create a prototype of visualization rather quickly. The first issue was dealing with the sheer number of nodes being created to represent all the lobbyists, clients, and agencies. I settled on just showing the top 50 paid lobbyists, but that still left me with a few nodes to show that were taking up a lot more vertical and horizontal space than I wanted. I address this by setting the layout’s charge to very repulsive and then setting the gravity to pull the nodes into the center. Unfortunately having these values set this way causes the node to bounce around when dragging is enabled you try to hover over a node. This seems like a bug and I plan on looking into what is causing that behavior.

Adjusting the charge and gravity that way organized the nodes into a circle and did a good job of spacing out the nodes and making the visualization take up less space. The problem was now that nodes were overlapping since certain nodes were larger than others. To solve this, I used the collision example from Mike Bostock’s presentation slides. Nodes no longer overlapped and the visualization was at a proper height and width.

Now, I had to make the connections of the individual nodes more visible and help the user to understand the data. I decided to do this by fading out all the nodes and links that are not connected to the hovered node. I struggled with how to efficiently traverse the nodes and adjust the opacity of the ones I wanted to fade out. I asked a question about the topic on stackoverflow about it and the creator of d3.js was kind enough to write a really helpful response that answered my question and gave me a better understanding of how to use the library.

The process of rendering all the starting frames while the graph reaches equilibrium was taking a lot of time. I found out that skipping the first few frames sped up the time to reach equilibrium and removed the distracting animation of the nodes bouncing all over the place. Initially, I was just skipping the first few frames, but I changed it to displaying a loading animation until the e.alpha cooling parameter reached a certain threshold.

Label Placement

Next, I had to decide how I would label the selected nodes. I toyed with a lot of ideas including force-based label placement, static placement of labels next to their node, and then placing the labels in a static position on the SVG and having a line point to the label it represents. Those solutions ended up being far too noisy and distracting. This is because there are too many nodes to put any sort of text or more lines next to each one. Finally, I settled on putting the labels in the top left corner and not having any lines or numbered labels to connect them to their nodes. Just color coding the node types and allowing the user to use the arrow keys to scroll through the nodes was an effective solution that didn’t clutter the screen more than was necessary.

Colors

Initially, the visualization had some really bad color choices. I tried to follow the red and blue color scheme used by ChicagoLobbyists.org, but there wasn’t enough color variance and contrast so it was difficult to see the contrast between nodes. I used ColorBrewer 2.0 to find a color scheme that would work for this and I was happy with the results.

Conclusion

Thanks to bl.ocks.org, I was able to quickly put the code on gist.github.com and share it with others. I am really happy with the overall results and the reception I got. I look forward to building more projects with d3.js and other libraries.

View the Visualization