Kafka engineer interview questions

As asked

You are designing an orders topic with 50,000 events per second. How do you choose the partition key, and what can go wrong?

Sample answer outline

The partition key controls ordering, load distribution and consumer parallelism, so it should match the ordering requirement first. If all events for an order must be processed in sequence, orderId is a natural key; if customer-level ordering matters, customerId may be required but can create hot partitions. The number of partitions should allow enough consumer parallelism without creating unnecessary broker overhead. The answer should discuss skew, key cardinality, compaction needs and future repartitioning pain. Candidates often say random keys for balance while missing that this destroys per-entity ordering.

Expect these follow-ups

• How would you detect a hot partition in production?
• What happens to ordering when you increase partition count?
• When would you split one logical stream across multiple topics?

As asked

Walk me through exactly what happens when a Kafka broker that holds a partition leader crashes. Who detects the failure, what mechanism picks the new leader, and what conditions can cause that election to fail or stall?

Sample answer outline

The controller broker monitors ZooKeeper (or KRaft metadata log) for broker liveness. On failure it selects the new leader from the in-sync replica set (ISR); if the ISR is empty and unclean.leader.election.enable is false, the partition goes offline. Strong answers distinguish ZooKeeper mode from KRaft and explain how controller epoch prevents split-brain.

Expect these follow-ups

• What happens if the ISR shrinks to zero replicas before the crash?
• How does KRaft change the controller election process compared to ZooKeeper mode?

As asked

A product manager says we need exactly-once delivery for our payment events. Explain exactly what Kafka's exactly-once semantics actually guarantee, the three moving pieces that make it work (idempotent producer, transactions, transactional consumer), and where it can still fail in practice.

Sample answer outline

Idempotent producers assign a producer ID (PID) and a per-partition sequence number so the broker deduplicates retries within a single producer session. Transactions (beginTransaction/commitTransaction) atomically write records across multiple partitions and then write a commit or abort control record to each involved partition so consumers know whether to include those records. Transaction state itself is persisted in the internal __transaction_state topic managed by the transaction coordinator. Consumers set isolation.level=read_committed to skip records belonging to aborted or in-flight transactions. Strong answers note that EOS does not cover external sinks and that the PID is lost on producer restart unless a transactional.id is configured, which allows the broker to fence any previous instance.

Expect these follow-ups

• What is the role of the transaction coordinator and __transaction_state topic?
• Why does EOS not automatically apply to Kafka Connect sinks?

As asked

When would you use auto-commit versus manual offset commits in a Kafka consumer, and what are the exact failure scenarios where auto-commit causes duplicate processing or data loss?

Sample answer outline

Auto-commit fires on a timer (auto.commit.interval.ms) and can commit offsets for records that have not finished processing, causing data loss if the consumer crashes. Manual commit with commitSync or commitAsync after processing guarantees at-least-once delivery. Strong answers describe the commit-after-poll vs commit-after-process distinction and using commitSync on shutdown for clean finalization.

Expect these follow-ups

• How do you implement exactly-once semantics using manual commits plus an idempotent sink?
• What is the danger of calling commitAsync without an error callback?

As asked

Your monitoring shows a Kafka consumer group with 50 million messages of lag on a critical topic and the lag is growing. Walk me through your diagnostic process: what metrics you check, what tooling you use, and what the three most likely root causes are.

Sample answer outline

Start with kafka-consumer-groups.sh --describe to see per-partition lag and which consumer is assigned each partition. Check consumer throughput (records-consumed-rate), broker throughput (bytes-in, bytes-out), and GC pause times. The three most common causes are slow message processing (CPU-bound or blocking I/O), under-partitioning (cannot scale horizontally), and a skewed partition causing one consumer to bottleneck. Strong answers also mention checking for rebalance storms via rebalance-rate metric.

Expect these follow-ups

• How do you safely add partitions to a topic that already has data?
• What tools would you use to replay messages from a specific offset after fixing the slow consumer?

As asked

A teammate says 'Kafka guarantees message order.' Correct or qualify that statement. Under what producer and topic configurations is ordering guaranteed, and name three common scenarios where ordering silently breaks.

Sample answer outline

Kafka guarantees order within a partition only. Ordering breaks when: messages with the same logical key are routed to different partitions (custom or null key), when a producer retries without idempotence (max.in.flight.requests.per.connection greater than 1 can cause reordering), and when a consumer reads from multiple partitions and merges them. Strong answers mention that idempotent producers enforce ordered delivery even with retries.

Expect these follow-ups

• How would you design a system that needs global ordering across all keys?
• What does max.in.flight.requests.per.connection=1 cost you in throughput?