At Facebook, we're constantly trying to make the world more open and connected. We don't just do this with the products we build, we also do it with the way we build products. Our site is built almost entirely with open source software, and we're committed to releasing the code we develop back to the community as open source. Open source software is code that is shared freely for anyone to use or improve. It's a great ideal as it keeps control of something important in the hands of the many instead of the few, but more importantly it turns out that being open is the best way to write good software. This week we're releasing a critical piece of our infrastructure software called scribe.

Scribe is a system for collecting massive amounts of data from a large number of servers, and we use it for everything from tracking how much memory a database is using to delivering relationship stories into News Feed. Before we wrote scribe, we tried a variety of open source and proprietary systems, but none of them could keep up with the massive amount of information our users were generating. So we wrote our own system, conceived from the start to handle the sorts of problems you only encounter at such large scale. It has turned out to be enormously useful, and we use it to move tens of billions of pieces of information every day.

If you'd like to dig deeper into the details of how Scribe works (or you have a lot of data to move), take a look at our engineering feed.


Bobby, director of infrastructure software, can't wait to see how much data you can move with Scribe.