Event-driven architecture lets services communicate by publishing events instead of requiring every consumer to be online at the exact same moment. Apache Kafka is a distributed event streaming platform commonly used when teams need durable, high-throughput streams that can be consumed independently by multiple services.
Event vs command
An event describes something that already happened: “OrderCreated” or “PaymentCaptured.” A command asks another component to do something. Keeping those meanings clear makes systems easier to reason about and prevents event topics from becoming disguised remote procedure calls.
Topics and partitions
Kafka stores records in topics, and topics are divided into partitions. Partitions provide the unit of parallelism and ordering. Records in the same partition are ordered, while records across different partitions should not be assumed to have one global order.
Producers
A producer writes records to a topic. Partitioning can use a key, which helps route related records to the same partition. For example, using customer ID as a key can preserve per-customer ordering while allowing different customers to be processed in parallel.
Consumers and consumer groups
Consumers read records. A consumer group shares the work of a topic across its members. When a member joins or leaves, partitions may be reassigned. Different groups can read the same events independently, allowing analytics, search indexing, notifications, and billing to evolve separately.
Offsets and replay
Kafka consumers track offsets so processing can resume. Because data can remain on disk for a configured retention period, a consumer can often replay older events. This is different from a transient queue where a message disappears as soon as it is acknowledged.
Delivery semantics
Exactly-once behavior is a system-level property, not a simple checkbox. Depending on the design, applications may use at-most-once, at-least-once, or stronger transactional patterns. At-least-once delivery is common, which means consumers should make important operations idempotent.
Failure handling
- Design consumers so retries do not duplicate irreversible side effects.
- Track processing failures separately from successful events.
- Monitor consumer lag, partition health, throughput, and error rates.
- Choose retention and partition counts based on workload rather than guesses.
When event-driven architecture helps
It is especially useful when many independent consumers need the same facts, when workloads should be decoupled in time, or when events need to be replayed for rebuilding a derived system. It also introduces operational complexity, so a simple synchronous API is often better for simple request-response workflows.
Kafka becomes easier to understand once you connect it to the bigger distributed system. Compare that model with ourconsistency and failure guideandscaling architecture deep dive.