This post explores why ordering is tricky in Kafka, and the strategies you can use to ensure order at scale.

Why Kafka Doesn’t Guarantee Global Ordering

Kafka maintains message order only inside a partition.

• If all events of a producer go into the same partition, order is preserved.

• Across multiple partitions, different consumers may process messages at different speeds, breaking the global order.

Example:
Imagine two consumers reading from two partitions. Kafka ensures partition-level order, but whichever consumer is faster will emit events first — leading to interleaved results.

Strategies to Preserve Ordering

1. Use a Partition Key

If you want related events (e.g., all events for a given Order ID) to be processed in sequence:

• Assign a partition key when producing the message.

• Kafka routes all events with the same key to the same partition.

• This ensures strict ordering for that entity.

Example:

ProducerRecord<String, String> record =
    new ProducerRecord<>("order-events", orderId, eventPayload);

Here, orderId acts as the partition key — ensuring all events for that order land in the same partition.

2. Assign a Dedicated Thread per Partition

Even if Kafka delivers events in order, consumer concurrency can break it.

• If your consumer processes records in parallel (e.g., using ExecutorService), order may be lost.

• Setting max.poll.records=1 preserves order but kills throughput.

A better approach is Thread per Partition:

Partition 1 → Thread 1  
Partition 2 → Thread 2  
Partition 3 → Thread 3

This way:

• Each partition is consumed by a dedicated thread.

• Ordering is maintained per partition.

• High throughput is preserved since partitions are processed independently.

Implementation idea:
Maintain a Map of partition → ExecutorService inside your listener:

• Key: Partition number

• Value: Thread pool dedicated to that partition

This design gives you the best of both worlds — throughput and ordering.

Key Takeaways

• Kafka guarantees order within partitions, not globally.

• Use partition keys to route related events to the same partition.

• Use Thread-per-Partition consumption to preserve order without sacrificing performance.

By applying these patterns, you can run Kafka at scale while still ensuring event ordering where it matters.

🔗 References:

• Kafka Message Key: A Comprehensive Guide

• Multi-Threaded Messaging with the Apache Kafka Consumer

• Java Techie: Kafka Ordering (YouTube)