Kafka Architecture Explained: Brokers, Topics, Partitions, and Offsets

Diagram: Kafka Components

Here’s a simple chart showing brokers, topics, partitions, and offsets.

graph TD
    subgraph Cluster["Kafka Cluster"]
        B1[Broker 1]
        B2[Broker 2]
    end

    subgraph TopicOrders["Topic: orders"]
        P0["Partition 0: [0,1,2,3...]"]
        P1["Partition 1: [0,1,2,3...]"]
        P2["Partition 2: [0,1,2,3...]"]
    end

    B1 --> TopicOrders
    B2 --> TopicOrders

Quick Notes

• 1 Cluster = many Brokers.
• 1 Broker = stores many Topics.
• 1 Topic = split into multiple Partitions.
• 1 Partition = ordered log with Offsets.

Conclusion

Kafka’s architecture may sound complicated, but it’s really just:

• Brokers: servers that store data.
• Topics: categories of messages.
• Partitions: split data for speed and scalability.
• Offsets: bookmarks to track where consumers are.

By combining these simple pieces, Kafka becomes a powerful platform for real-time data.