Dec. 14, 2010 at 1:13 PM ET
With 500 million users, Facebook is as big as the population of the European Union.
So a Facebook intern has devised a visual way to interpret at least one aspect of all the potential data in so many users: "the locality of friendship."
Paul Butler, an intern on Facebook’s data infrastructure engineering team, took it upon himself to use R, an open-source environment for statistical computing and graphics. He explained his jumping point this way:
I was interested in seeing how geography and political borders affected where people lived relative to their friends. I wanted a visualization that would show which cities had a lot of friendships between them. I began by taking a sample of about ten million pairs of friends from Apache Hive, our data warehouse. I combined that data with each user's current city and summed the number of friends between each pair of cities. Then I merged the data with the longitude and latitude of each city.
Butler plotted points at some of the latitude and longitude coordinates and came up with a rough outline of the world. He plotted lines between the points. A "big white blob" appeared in the center of his map, and while some of the outer edges shared a vague resemblance to continents, it soon became apparent that he had "too much data to get interesting results just by drawing lines." Other graphing techniques also went down the drain: semi-transparent lines, too many shades of color.
He explains how he reached his desired effect:
I defined weights for each pair of cities as a function of the Euclidean distance between them and the number of friends between them. Then I plotted lines between the pairs by weight, so that pairs of cities with the most friendships between them were drawn on top of the others. I used a color ramp from black to blue to white, with each line's color depending on its weight. I also transformed some of the lines to wrap around the image, rather than spanning more than halfway around the world.
The desired effect: No longer a blob, but this detailed map of the world seen above, including continents and some borders. Butler couldn't help but be effusively sentimental about what he saw.
What really struck me, though, was knowing that the lines didn't represent coasts or rivers or political borders, but real human relationships. Each line might represent a friendship made while travelling, a family member abroad, or an old college friend pulled away by the various forces of life.
Butler replaced the lines with circle arcs, which are the shortest routes between two points on the Earth.
I still can't wrap my head around all this, but the image is a gorgeous rendering of one way Facebook interprets friendships: points of light. That, I can definitely understand and agree with.
Or, the way Butler sees it: "It's not just a pretty picture, it's a reaffirmation of the impact we have in connecting people, even across oceans and borders."
What's it mean to you?