The Facebook Blog

We've talked before about the massive amount of data running through Facebook's systems. We have Lexicon to help visualize what people are talking about on the site, but we don't yet have any similar, graphics-driven way to show the site's constant activity and connections. Taking a break from my normal job (if dealing with Facebook's massive scalability challenges counts as normal) at the most recent hackathon, I decided to brush off my graphics abilities and work on something more visual.

I focused on a project known as Palantir. We're... still experimenting with it, but the idea is to show Facebook in action, with people's interactions speeding across a computer model of the earth, as if friend requests, messages or photo-sharing exchanges were bursts of light or airplanes flying to their destinations.

Palantir was started at a previous hackathon with basic visualizations for actions. After an all-night coding binge, the framework now has the potential to visualize many kinds of data: social interaction, the effectiveness of the Facebook advertising model, or even actions in your own network of friends.

It's also another example of our commitment to using open-source software, in which public access to the underlying code allows a wide range of people to make improvements. On the server side, it filters data that Facebook generates daily, accessed using a combination of our open source protocols Thrift and Scribe. The client is written in Java using the open source jME framework, which connects to the Palantir server and streams this data live.

You can see the video demo of the first iteration of Palantir, which shows 3 1/2 minutes of Facebook activity this way. New ties are forming every second between the United States and Latin America; between Europe and Australia; and in fact between almost any two parts of the planet. Seeing the data come to life shows how our world is becoming more and more connected.


Jack Lindamood wishes we could get frequent-flyer miles for Palantir's activity.