EP82: Open-sourcing Over 100 Byte-sized System Design Concepts

This week’s system design refresher:

• Open-sourcing over 100 byte-sized system design concepts with high-resolution diagrams

• Best ways to test system functionality

• Cloud Network Components Cheat Sheet

• Explaining 5 unique ID generators in distributed systems

---

Organize your API work and collaborate more (Sponsored)

Postman workspaces give teams shared access to the tools they need to solve problems together. They are the go-to place for development teams to collaborate and move quickly while staying on the same page.

With workspaces, teams can:

• Automatically notify other team members about changes to APIs as updates sync in real time.

• Set up manual or automated workflows to support different stages of API development.

• Enable faster onboarding for both internal and external partner developers

• Create collaborative hubs for troubleshooting API calls and maintaining a log of common steps to follow.

---

Open-sourcing over 100 byte-sized system design concepts with high-resolution diagrams

Start exploring the repository here: https://bit.ly/bytebytegoGitRepo

---

Best ways to test system functionality

Testing system functionality is a crucial step in software development and engineering processes.

It ensures that a system or software application performs as expected, meets user requirements, and operates reliably.

Here we delve into the best ways:

1. Unit Testing: Ensures individual code components work correctly in isolation.

2. Integration Testing: Verifies that different system parts function seamlessly together.

3. System Testing: Assesses the entire system's compliance with user requirements and performance.

4. Load Testing: Tests a system's ability to handle high workloads and identifies performance issues.

5. Error Testing: Evaluate how the software handles invalid inputs and error conditions.

6. Test Automation: Automates test case execution for efficiency, repeatability, and error reduction.

Over to you: How do you approach testing system functionality in your software development or engineering projects?

---

Latest articles

If you’re not a subscriber, here’s what you missed this month.

1. The 6 Most Impactful Ways Redis is Used in Production Systems

2. The Tech Promotion Algorithm: A Structured Guide to Moving Up

3. A Crash Course in DNS

4. A Crash Course in Redis

5. Why is Kafka so fast? How does it work?

To receive all the full articles and support ByteByteGo, consider subscribing:

---

Cloud Network Components Cheat Sheet

Network components form the backbone of cloud infrastructure, enabling connectivity, scalability, and functionality in cloud services.

These components include routers, load balancers, and firewalls, which ensure data flows efficiently and securely between servers and clients.

Additionally, Content Delivery Networks (CDNs) optimize content delivery by caching data at edge locations, reducing latency and improving user experience.

In essence, these network elements work together to create a robust and responsive cloud ecosystem that underpins modern digital services and applications.

This cheat sheet offers a concise yet comprehensive comparison of key network elements across the four major cloud providers.

Over to you: How did you tackle the complexity of configuring and managing these network components?

---

Explaining 5 unique ID generators in distributed systems

The diagram below shows how they work. Each generator has its pros and cons.

1. UUID
   A UUID has 128 bits. It is simple to generate and no need to call another service. However, it is not sequential and inefficient for database indexing. Additionally, UUID doesn’t guarantee global uniqueness. We need to be careful with ID conflicts (although the chances are slim.)

2. Snowflake
   Snowflake’s ID generation process has multiple components: timestamp, machine ID, and serial number. The first bit is unused to ensure positive IDs. This generator doesn’t need to talk to an ID generator via the network, so is fast and scalable.
   Snowflake implementations vary. For example, data center ID can be added to the “MachineID” component to guarantee global uniqueness.

3. DB auto-increment
   Most database products offer auto-increment identity columns. Since this is supported in the database, we can leverage its transaction management to handle concurrent visits to the ID generator. This guarantees uniqueness in one table. However, this involves network communications and may expose sensitive business data to the outside. For example, if we use this as a user ID, our business competitors will have a rough idea of the total number of users registered on our website.

4. DB segment
   An alternative approach is to retrieve IDs from the database in batches and cache them in the ID servers, each ID server handling a segment of IDs. This greatly saves the I/O pressure on the database.

5. Redis
   We can also use Redis key-value pair to generate unique IDs. Redis stores data in memory, so this approach offers better performance than the database.

Over to you - What ID generator have you used?