Why the Internet Is Both Robust and Fragile

The Internet blinked and Facebook disappeared. For nearly six hours in October 2021, Facebook’s apps vanished from the online world. In this issue, we unpack the anatomy of this severe outage to understand the deeper machinery powering the global Internet.

• Recap how DNS resolution works, from root servers to resolving a domain

• Explain the border gateway protocol (BGP) - the glue holding the decentralized Internet together

• Trace how packets traverse multiple networks to reach destinations

• Reconstruct the sequence of events that caused Facebook's infrastructure to vanish

• Discuss the power and fragility of the core protocols that keep information flowing online

By peering into the inner workings of DNS and BGP, we’ll shed light on the incredible infrastructure that keeps us all connected online. Let's dive into the fascinating world of protocols behind the scenes.

In a recent issue, we offered an in-depth exploration of DNS. This week, let’s take a step further. We’ll look at how the Internet itself is put together, using what we learned about DNS as an illustration.

To bring these abstract concepts to life, we’ll take a closer look at the Facebook global outage in October 2021 caused by their DNS servers disappearing from the map. So buckle up as we dive deeper into the mechanisms that keep us all connected online.

---

DNS Recap

Let’s quickly recap what we learned about DNS from our recent issue.

IP Addresses

We start with the concept that every computer on the Internet has a numerical IP address that it uses to communicate.

There are two types of IP addresses.

IPv4: A 32-bit number written out as four 8-bit numbers separated by dots (e.g., 192.168.1.1)

IPv6: A 128-bit number written in hexadecimal format (e.g. 2001:db8::1)

The internet is slowly moving towards adopting IPv6 more broadly because it supports a much bigger address space.

For simplicity, in this issue, we’ll focus on IPv4, since the dot-separated numbers are easier to work with for illustrative examples.

Now that we've covered the IP address basics, let's recap how DNS helps map domain names to these underlying IP addresses.

DNS Resolution Step-by-Step

When we type a domain name like blog.bytebytego.com into the browser, the computer has to look up the IP address for the name.

This is accomplished using DNS, the domain name system we explored in-depth last week.

Let’s recap how it works: The computer starts by contacting a recursive resolver, typically one run by the ISP or a major provider like Google’s 8.8.8.8 and Cloudflare’s 1.1.1.1.

The recursive resolver then queries the DNS hierarchy on our behalf, starting with the root servers.

These root DNS servers have well-known public IP addresses maintained by the Internet Assigned Numbers Authority. There are 13 IP addresses worldwide.

The servers are maintained and operated by different entities worldwide. They are corporations, educational institutions, governments, and various nonprofits. There is no single entity that controls all these root servers.

These 13 IP addresses span across about 1700 server instances as of this writing. That means each IP address is served by more than a single server. As we progress through this issue, you’ll learn more about the techniques that make it possible.

Everyone agrees these 13 IP addresses are the authoritative root servers. We can send a request to any one of them to look up an IP address.

Let’s go over an example of performing a DNS lookup from one of the root servers. We can use the dig command to ask a root server to resolve blog.bytebytego.com:

The IP address 202.12.27.33 is the root server m.root-servers.net. It returned “ANSWER: 0” - it doesn’t have the IP address for our target domain. But it provided “AUTHORITY: 13” - the 13 “.com” name servers it knows about.

As we learned from the last issue, the root servers do not hold the actual IP addresses for our target domain “blog.bytebytego.com”. Instead of providing an answer, it directed us to the 13 .com TLD name servers, as shown here:

It even gave us their IP addresses under the “Additional Section”. We’ll show a subset here:

The root server is saying it doesn’t have the answer, but to talk to one of the .com servers. So let’s ask one about blog.bytebytego.com. We’ll ask 192.43.172.30:

Again, no final answer, but we’re closer! It returned 2 authoritative name servers for bytebytego.com and their IP addresses:

Now, let’s query one of those authoritative servers:

Now we have the CNAME record pointing to the canonical name. We could continue querying to get the final IP address. We’ll leave it as an exercise for the readers.

How DNS Resolvers Simplify the Process

When the computer requests the IP address for “blog.bytebytego.com”, it offloads the lookup task to a DNS resolver.

If we run dig without specifying a server, it uses my configured resolver, which is usually maintained by the ISP. It directly returns three answers: