EP126: The Ultimate Kafka 101 You Cannot Miss
This week’s system design refresher:
How the Internet Works in 9 Minutes (Youtube video)
AWS Services Cheat Sheet
The Ultimate Kafka 101 You Cannot Miss
8 Tips for Efficient API Design
SPONSOR US
✂️Cut your QA cycles down to minutes with automated testing (Sponsored)
Are slow test cycles bottlenecking your dev teams’ release velocity? With QA Wolf, your organization can run entire test suites in minutes for faster feedback and developer confidence to ship.
QA Wolf takes testing off your plate. They can get you:
80% automated test coverage in weeks—not years
24-hour maintenance and on-demand test creation
Zero flakes, guaranteed
The benefit? No more manual E2E testing. No more slow QA cycles. No more bugs reaching production.
With QA Wolf, Drata’s team of 80+ engineers achieved 4x more test cases and 86% faster QA cycles.
🌟Rated 4.8/5 on G2.
How the Internet Works in 9 Minutes
AWS Services Cheat Sheet
AWS grew from an in-house project to the market leader in cloud services, offering so many different services that even experts can find it a lot to take in.
The platform not only caters to foundational cloud needs but also stays at the forefront of emerging technologies such as machine learning and IoT, establishing itself as a bedrock for cutting-edge innovation. AWS continuously refines its array of services, ensuring advanced capabilities for security, scalability, and operational efficiency are available.
For those navigating the complex array of options, this AWS Services Guide is a helpful visual aid.
It simplifies the exploration of AWS's expansive landscape, making it accessible for users to identify and leverage the right tools for their cloud-based endeavors.
Over to you: What improvements would you like to see in AWS services based on your usage?
Latest articles
If you’re not a paid subscriber, here’s what you missed.
To receive all the full articles and support ByteByteGo, consider subscribing:
The Ultimate Kafka 101 You Cannot Miss
Kafka is super-popular but can be overwhelming in the beginning.
Here are 8 simple steps that can help you understand the fundamentals of Kafka.
What is Kafka?
Kafka is a distributed event store and a streaming platform. It began as an internal project at LinkedIn and now powers some of the largest data pipelines in the world in orgs like Netflix, Uber, etc.Kafka Messages
Message is the basic unit of data in Kafka. It’s like a record in a table consisting of headers, key, and value.Kafka Topics and Partitions
Every message goes to a particular Topic. Think of the topic as a folder on your computer. Topics also have multiple partitions.Advantages of Kafka
Kafka can handle multiple producers and consumers, while providing disk-based data retention and high scalability.Kafka Producer
Producers in Kafka create new messages, batch them, and send them to a Kafka topic. They also take care of balancing messages across different partitions.Kafka Consumer
Kafka consumers work together as a consumer group to read messages from the broker.Kafka Cluster
A Kafka cluster consists of several brokers where each partition is replicated across multiple brokers to ensure high availability and redundancy.Use Cases of Kafka
Kafka can be used for log analysis, data streaming, change data capture, and system monitoring.
Over to you: What else would you add to get a better understanding of Kafka?
8 Tips for Efficient API Design
Domain Model Driven
When designing the path structure of a RESTful API, we can refer to the domain model.Choose Proper HTTP Methods
Defining a few basic HTTP Methods can simplify the API design. For example, PATCH can often be a problem for teams.Implement Idempotence Properly
Designing for idempotence in advance can improve the robustness of an API. GET method is idempotent, but POST needs to be designed properly to be idempotent.Choose Proper HTTP Status Codes
Define a limited number of HTTP status codes to use to simplify application development.Versioning
Designing the version number for the API in advance can simplify upgrade work.Semantic Paths
Using semantic paths makes APIs easier to understand, so that users can find the correct APIs in the documentation.Batch Processing
Use batch/bulk as a keyword and place it at the end of the path.Query Language
Designing a set of query rules makes the API more flexible. For example, pagination, sorting, filtering etc.
SPONSOR US
Get your product in front of more than 1,000,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing sponsorship@bytebytego.com
One small feedback, what ever your title is that should be at top most level, and we do not want other articles or other stuff. We expect something in details, and what we get is other non-related stuff to what title it indicates, it just disrupts the flow. Better would be to include only one article and links to other articles that would make sense
This is a fantastic article; the graphics are superb! How did you make the animations? I would like to note that most servers and internet hardware run Linux, rather that Mac OS or Windows. If you would like to know more, read this:
https://swiftenterprises.substack.com/p/live-free-with-linux
Live Free with Linux!