Netflix: What Happens When You Press Play - Part 2

Welcome back! This is Part 2 of "Netflix: What Happens When You Press Play?” by guest author, Todd Hoff.

For those interested in a deeper dive into other subjects of the book Explain the Cloud Like I’m 10, check it out here.

---

Remember how we said a CDN has computers distributed all over the world? 

Netflix developed its own computer system for video storage. Netflix calls them Open Connect Appliances or OCAs. 

Here’s what an early OCA installation in a site looked like:

OCA Site from Netflix

There are many OCAs in the above picture. OCAs are grouped into clusters of multiple servers.

Each OCA is a fast server, highly optimized for delivering large files, with lots and lots of hard disks or flash drives for storing video. 

Here’s what one of the OCA servers looks like:

Open Connect Appliance from Netflix

There are several different kinds of OCAs for other purposes. There are large OCAs that can store Netflix’s entire video catalog. Some smaller OCAs can store only a portion of Netflix’s video catalog. Smaller OCAs are filled with video daily, during off-peak hours, using a process that Netflix calls proactive caching. We’ll talk more about how proactive caching works later.

From a hardware perspective, there’s nothing special about OCAs. They’re based on commodity PC components and assembled in custom cases by various suppliers. You could buy the same computers if you wanted to.

Notice how all of Netflix’s computers are red? Netflix had their computers specially made to match their logo color.

From a software perspective, OCAs use the FreeBSD operating system and NGINX for the web server. Yes, every OCA has a web server. Video streams using NGINX. If none of these names make sense, don’t worry, I just include them for completeness.

The number of OCAs on a site depends on how reliable Netflix wants the site to be, the amount of Netflix traffic (bandwidth) that is delivered from that site, and the percentage of traffic a site allows to be streamed. 

When you press play, you’re watching video streaming from a specific OCA, like the one above, in a location near you. 

For the best possible video viewing experience, what Netflix would like to do is cache video in your house. But that’s not practical yet. The next best thing is to put a mini-Netflix as close to your home as possible. How do they do that?

Where does Netflix put Open Connect Appliances (OCAs)?

Netflix delivers vast amounts of video traffic from thousands of servers in more than 1,000 locations worldwide. Take a look at this map of video serving locations: 

CDN Locations from Netflix

Other video services, like YouTube and Amazon, deliver video on their own backbone network. These companies built their own global network for delivering video to users. That’s very complicated and very expensive to do. 

Netflix took a completely different approach to build its CDN.

Netflix doesn’t operate its network; it doesn’t operate its own data centers anymore. Instead, internet service providers (ISPs) agree to put OCAs in their data centers. OCAs are offered free to ISPs to embed in their networks. Netflix also puts OCAs in or close to internet exchange locations (IXPs). 

Using this strategy, Netflix doesn’t need to operate its own data centers, yet it gets all the benefits of being in a regular datacenter. It’s just someone else’s datacenter. Genius!

Those last two paragraphs were dense, so let’s break it down. 

Using ISPs to build a CDN.

An ISP is your internet provider. It’s where you get your internet service from. It might be Verizon, Comcast, or thousands of other services. 

The main point here is that ISPs are located all around the world, and they’re close to customers. By placing OCAs in ISP data centers, Netflix is also worldwide and close to its customers.

Using IXPs to build a CDN.

An internet exchange location is a datacenter where ISPs and CDNs exchange internet traffic between their networks. It’s like going to a party to exchange Christmas presents with your friends. It’s easier to exchange gifts if everyone is in one place. It’s easier to exchange network traffic if everyone is in one place. 

IXPs are located all over the world:

TeleGeography's Internet Exchange Map