Client-Cluster Communication Flow

Clients never talk to your service directly. They talk to the cluster. That indirection is the whole point — it is what turns a stream of client requests into an ordered, replicated, durable log before your business logic ever sees a byte.

This page traces a message end to end: connect, redirect to the leader, ingress, consensus, service execution, egress. Knowing where each hop happens tells you exactly where p50, p99, and throughput are won or lost.

The model in one line

Clients go through a well-defined protocol — connect, order, replicate, execute, respond — not a direct call to your service. Every message is ordered and replicated before the service sees it. That is where the consistency guarantees come from.

The protocol at a glance

A client message follows four phases. Each phase has a job, and each adds latency you can reason about.

1. Client connects to any cluster node — and may be redirected to the leader.
2. Messages flow through the consensus module — ordered and replicated.
3. Service processes committed messages — deterministic execution.
4. Responses flow back through egress — leader to client.

The indirection is the feature. Because every message is ordered and replicated before execution, the service sees a single, agreed-upon stream — identical on every node.

Who’s in the conversation

These are the participants in the full sequence. Keep them in mind; the latency budget is spread across all of them.

Participant	Role
Aeron Cluster Client	Your application’s client library
Aeron Cluster Leader Node	The node that accepts writes
Aeron Cluster Member Node (Follower)	Replicates the log
Consensus Module	Raft implementation; orders and replicates entries
Aeron Archive	Persists log entries to disk
Service Container	Hosts your clustered service
Customer Built Cluster Logic	Your business code (matching engine, etc.)

Defer the wire details

This page covers the flow. For how the consensus module, archive, and media driver move bytes under the hood, see The Aeron Files. We link rather than duplicate.

The full calling sequence (24 steps)

The complete journey is four phases. Read it top to bottom and the latency story tells itself.

Phase 1 — Client connection (steps 1-6)

A client may connect to any node. If that node is a follower, it gets redirected to the leader. Then the connection itself is logged.

The takeaway: even connecting goes through the log. The session is durable from the first handshake.

Phase 2 — Request processing (steps 7-14)

This is the consensus hop. The leader appends the request locally and replicates it to followers in parallel. Commit happens only when a quorum acknowledges.

This phase dominates the latency budget. The leader counts itself toward the quorum (quorumThreshold = n/2 + 1, ClusterMember.java), so in a 3-node cluster commit needs just one follower to have appended to its archive and ACKed back. That round trip — replicate, persist, acknowledge — is where p99 lives.

Quorum sets your p99 floor

Commit waits for the fastest follower, not the slowest member: with 3 nodes, quorum = 2 = leader + 1 ACK (quorumPosition() commits at the 2nd-highest log position). So one slow follower does not drag commit latency — the cluster rides on whichever follower answers first. The tail appears when slowness is correlated (both followers GC together, shared AZ jitter), and a persistently slow follower has a subtler cost: it silently removes your redundancy — lose the healthy follower and every commit now waits on the laggard. That’s why you tune both followers, not just the leader.

Phase 3 — Service execution (steps 15-17)

Only committed log fragments reach the service. Execution is deterministic — same input, same state change, on every node.

Your matching engine runs here. Because it only ever sees committed, ordered messages, it never has to reason about consensus, retries, or replication. Keep onSessionMessage() tight: this is in the hot path, and any allocation or blocking call here shows up directly in p50.

Phase 4 — Response to client (steps 18-24)

The response is not a shortcut back to the client. It goes through consensus too — logged, replicated, committed, then delivered.

So a single request crosses consensus twice — once on the way in, once on the way out. That is the price of exactly-once semantics and clean replay.

Key Insight: responses are durable too

The response path (steps 18-24) also goes through consensus — the response is logged and replicated before being delivered to the client. This means responses are also part of the durable log, which matters for exactly-once semantics and replay.

The entire flow is:

ingress → log → replicate → commit → execute → egress → log → replicate → commit → deliver.

Every step is ordered and durable.

What this means for tuning

Map the phases to your latency targets and the priorities fall out:

• p99 is mostly Phase 2 and Phase 4 — the two consensus round trips. Faster follower disks and lower-jitter inter-node links move p99 more than anything you do on the leader alone.
• p50 is mostly Phase 3 — your onSessionMessage() logic. Determinism plus a tight hot path keeps the median low.
• Throughput is set by the consensus module’s ability to batch — the more requests it can append and replicate per round trip, the higher the ceiling. Ingress and egress are both gated by it.

Because every request pays the consensus tax twice, the cheapest win is almost always making the quorum round trip faster — not optimizing the service code that runs once in the middle.