Every day's a good day for taking and sharing photos, but at Facebook, Halloween is the best of all. Our users go all out to capture the zany costumes and freewheeling spirit, typically uploading about 20% more photos than usual during Halloween week. That can include several daily spikes that are more than double normal levels. Last year, we welcomed 124 million photo uploads during that week, and this year we're expecting a lot more.

To handle all that activity, we've added 20% more photo servers and 50% more upload servers to process this year's Halloween traffic. We've also packed in 40 terabytes of additional storage (that's 40,000,000,000,000 bytes). By comparison, the words of all 20 million books in the Library of Congress could be digitized in about 20 terabytes of text. So we're making room for the equivalent of two Libraries of Congress.

The popularity of Facebook's photo application, home of more than 10 billion photos, has compelled us to think big for a while. We calculated that if all our photos were printed out and placed side by side, they would occupy 10,280 square miles. That's bigger than Jamaica. Or, if by some bit of magic you could connect all the pixels of our photos into a very long, one-pixel-wide string, it would stretch from the sun to partway between the orbits of Jupiter and Saturn. We'd share some more weird statistics, but it's time to find a good costume.

Facebook users love our photos application, it's one of the most popular areas on the site. The cool front-end features—like upload, tagging, and editing—are great, but I'd like to take a moment and talk about our photo infrastructure. It's the part of our photos feature that users never see, but it quietly works 24 hours a day to quickly serve all those photos to millions of users.

Allow me to geek out and share some photo infrastructure statistics with
you. We have:

• 1.7 billion user photos


• 2.2 billion friends tagged in user photos


• 160 terabytes of photo storage used with an extra 60 terabytes available


• 60+ million photos added each week which take up 5 terabytes of disk space


• 3+ billion photo images served to users every day


• 100,000+ images served per second during our peak traffic windows

(For you sharp-eyed geeks out there, we actually store four image sizes for each photo, so if you want to know how many files we have, multiply the photo numbers by four. That's right, we're adding a quarter billion files every week.)

And that's just the storage side of things. We also have a fleet of servers who receive your photo uploads, happily scrubbing out EXIF data and tweaking the quality settings so your photos look their best. Another fleet of servers and caches who feed images to our CDN (content delivery network) partners. And hey, if we feel like it, we can serve some of the photo traffic directly ourselves too.

We have also developed our own specialized web servers that are tuned to serve files with as few disk reads as possible. Even with thousands of hard drive spindles, I/O (input/output) is still a concern because our traffic is so high. Our squid caches help reduce the load, but squid isn't nearly fast or efficient enough for our purposes, so we're writing our own web accelerator too.

That's just a taste of the current photo infrastructure, and while the system we've got is pretty sweet, there are even bigger things to come. Browser-side sampling of image response times so we can immediately know when ISPs are having issues, automated balancing of load between CDNs and the Facebook network (why pay a CDN to serve images in the middle of the night when we've got a huge photo serving tier that's quiet at that time?), better image performance for our international users, the list goes on and on...



Doug has a bet with the storage team that Facebook will have a petabyte of photos before April 2008, so please don't let him down.