app
→ events · 0/s · acks=all
No keys (sticky)
sent 0 · acked 0 · failed 0
Kafka, live
Three brokers, two topics, two consumer groups, all running right here. Take a guided scenario, or break things yourself: stop a broker, cut its power, add a consumer, drag the timeline back to see it again, or type kafka-topics --describe in the terminal.
KIP-101, part 2: the same restart with leader epochs (Kafka 0.11+)
Step 1 of 5
Same start
The same two brokers and the same producer, but this cluster uses Leader epochA counter that goes up with every new leader. Each record carries the epoch it was written in, so a returning replica can ask where its epoch ended and cut off exactly what diverged (KIP-101). for truncation, like every Kafka since 0.11.
Based on KIP-101 (Apache Kafka)
How to read this picture
Brokers
- Power light. Green: running. Red: stopped.
- Activity light. Blinks while records go in or out.
- The KRaft controller. It decides which replica leads each partition.
- In and Out: network bytes per second. Disk: bytes in its log files. Cache: bytes written but not yet on disk.
- Dashed border: a slow broker. Its followers fall behind.
- Grey stripes: a stopped broker.
- Leader replica: takes writes and serves reads
- Follower replica: copies the leader
- Out of the ISR: too far behind to count
- Number on the right: the log end offset, the next offset this replica will write.
A partition’s log
- A record; the colour is its key
- No key
- Striped: only in the page cache, lost if the power goes
- Faded: above the high watermark, not yet on every ISR replica, so consumers can’t read it
- Red inner border: a tombstone (null value) that deletes its key in a compacted topic
- Grey hatching: deleted by retention or compaction
- Dotted: an offset this replica doesn’t have yet
- Red outline: differs from the leader at the same offset
- High watermark: consumers read only below this line
- A follower’s own idea of the high watermark, which lags the leader’s
- Blue mark (in Inspect): where a consumer group will read next
- Gold frame: the partition you picked. The Inspect panel below shows it.
Producers and consumers
- Envelope: a producer. acks says how many replicas must have a record before it counts as written.
- Dashed producer: records are waiting in it because no leader can be reached.
- Face: one consumer in a group. Lag is how many records it hasn’t read yet.
- Group: its state (Stable, rebalancing…), assignor and generation, which goes up at every rebalance.
What moves
- Solid square: a record on its way from a producer to the leader (colour = key)
- Hollow square: a follower copying records from the leader
- Round dot: records fetched by a consumer
- Crossed square: a request that failed
Producers
Cluster
Broker 1
ControllerRunning
In0 B/s
Out0 B/s
Disk0 B
Cache0 B
Broker 2
Running
In0 B/s
Out0 B/s
Disk0 B
Cache0 B
Consumer groups
No consumer groups yet.
events · partition 0
- Leader
- 1
- Leader epoch
- 0
- Replicas
- 1, 2
- In-sync replicas
- 1, 2
- Log start offset
- 0
- High watermark
- 0
- Log end offset
- 0
Click a record to follow its journey.
Segment files on broker 1 Leader
00000000000000000000.log0 records0 B.index .timeindex
Segment files on broker 2 ISR
own HW 0
00000000000000000000.log0 records0 B.index .timeindex
HW: consumers read below this line · LEO: next offset to write · only in the page cache · differs from the leader at the same offset
NOTES
Kafka notes
The concepts behind what you see above. They also live under Notes, tagged #kafka.
- Build your own Kafka, 0: why a logBefore writing any code: why Kafka stores messages in an append-only log instead of a queue, and what that one decision buys.
- Build your own Kafka, 1: records on diskThe exact bytes Kafka writes for a batch of records: a 61-byte header, varint-encoded records and a CRC-32C. You write the varint encoder.
- Build your own Kafka, 2: offsets and the indexA consumer asks for offset 217. How a broker finds it in a gigabyte file without scanning: a sparse index, binary search, then a short scan. You write the lookup.
- Build your own Kafka, 3: segments and retentionAn append-only file can only grow. Splitting the log into segment files makes deleting old data cheap. You write the retention rule.
- Build your own Kafka, 4: the page cache and crash recoveryKafka does not fsync each write. What that costs when the power goes, how a broker repairs a torn log on startup, and why replication is the real answer. You write the recovery.
- Kafka: consumer groups, rebalances and lagHow a group splits partitions between its members, what happens when someone joins, leaves or crashes, where committed offsets live, and how to read lag.
- Kafka: topics, partitions, offsets and the logA topic is a set of append-only logs. How records get an offset, which partition a key lands on, and what the files on a broker’s disk look like.
- Logging in to Kafka and MySQL from a laptop with a client certificateA private CA, a client certificate, an SSH tunnel, and DataGrip: the command to run every time, the one-time setup, and every question that came up on the way.
- Kafka: replication, the ISR and the high watermarkLeaders and followers, which replicas count as in sync, what acks=all really waits for, and how Kafka avoids (or allows) losing writes when a broker dies.