Bandwidth Delay Product (BDP)

BDP is the single most important number for sizing Aeron transport. Get it right and the pipe stays full. Get it wrong and you either cap throughput or bloat your tail latency.

Read this alongside the tuning overview and the parameter reference.

The Formula

BDP is the maximum amount of data that can be “on the wire” at any given time.

Bytes in Flight = Bandwidth × Round Trip Latency

Bandwidth	RTT	Bytes in Flight
10 Gbit/s	100 µs (same AZ)	0.125 Mbit
10 Gbit/s	1 ms (cross AZ)	1.25 Mbit

Why This Matters for Aeron

BDP tells you the maximum amount of data that can be “on the wire” at any given time. It derives from Little’s Law (L = λW):

• L — number of items in the system (bytes in flight)
• λ — arrival rate (bandwidth)
• W — time in system (round-trip latency)

For how Aeron’s flow control and term buffers actually move these bytes, defer to The Aeron Files. This page stays focused on sizing.

Buffer Sizing Rule of Thumb

Size buffers above the BDP, but not wildly above it.

Rule of thumb

Buffers should be sized at 2x–4x the Bandwidth Delay Product.

The trade-off is direct:

• Too small (below BDP) leaves bandwidth on the table. The sender waits for ACKs before it can send more, so throughput drops.
• Too large risks bufferbloat. Deep queues inflate p99 and increase tail latency.
• Right-sized (near BDP, with headroom) keeps the pipe full while holding p50 and p99 steady.

The Effect of BDP

Two relationships drive every decision here.

Buffer size vs. RTT under constant TPS

Buffer Size
    ↑
    │            ╱ Cross Region
    │          ╱
    │        ╱ Cross AZ
    │      ╱
    │    ╱ Same AZ
    │  ╱
    │╱ CPG (Cluster Placement Group)
    └──────────────────→ RTT

As RTT increases (CPG → Same AZ → Cross AZ → Cross Region), the minimum required buffer size grows linearly with RTT to hold the same throughput.

TPS under constant buffer size

With a fixed buffer size, TPS drops as RTT increases — because the pipe can’t stay full.

Worked example: BDP from SBE size, TPS, and RTT

The table further below sizes from link bandwidth — the upper bound. But what you usually know is your business load: an SBE-encoded message size and a target TPS. Convert that to bandwidth first, then apply the BDP formula:

on-wire bytes/msg = align32( 32-byte Aeron data header + SBE encoded length )
bandwidth λ       = TPS × on-wire bytes/msg
BDP               = λ × RTT
window            = 2–4 × BDP   (and never below the 128KB default)

The 32-byte header and 32-byte frame alignment come from the Aeron protocol (DataHeaderFlyweight.HEADER_LENGTH, FrameDescriptor.FRAME_ALIGNMENT). The SBE encoded length is the 8-byte SBE message header plus your fields (block + variable-length data).

Example — an order-entry stream between two cluster nodes:

Step	Calculation	Result
SBE encoded length	8-byte SBE header + 88 bytes of order fields	96 B
On-wire per message	align32(32 + 96)	128 B
Bandwidth at 500k TPS	128 B × 500,000/s	64 MB/s (~0.5 Gbps)
BDP, same AZ (RTT 100 µs)	64 MB/s × 0.0001 s	6.4 KB
BDP, cross AZ (RTT 1 ms)	64 MB/s × 0.001 s	64 KB
BDP, cross region (RTT 50 ms)	64 MB/s × 0.05 s	3.2 MB

Then apply the 2–4× rule and the receive-path constraints:

• Same/cross AZ: 2–4× BDP is 128–256 KB — the 128KB default window already covers same-AZ, and cross-AZ needs at most one bump. Defaults are fine; this is why same-AZ testing never hurts.
• Cross region: 2–4× BDP is 6.4–12.8 MB. Now the chain reacts: window 12.8 MB ⇒ term buffer ≥ 2 × window = 25.6 MB ⇒ round up to 32 MB (power of 2) — the 16MB default term is too small. Set SO_RCVBUF (and rmem_max) ≥ the window too.

Measure RTT with the shipped samples (io.aeron.samples.Ping / Pong) for the application-level number, or ping between nodes for the network floor.

TPS-derived BDP is the floor, not the ceiling

Steady-state traffic flows at your TPS, but recovery does not: NAK retransmits, a late-joining subscriber, or archive replay run at link rate. If buffers are sized only for the TPS-derived BDP, a recovering stream stalls on flow control exactly when you need it to catch up fastest. Size between the TPS-derived floor and the link-rate numbers below according to how fast you need recovery to be.

Two refinements to the per-message math: if you batch messages into MTU-sized frames (smart batching), the 32-byte header amortizes across the batch, so the unbatched figure above is conservative — safe for sizing. And if your messages carry large variable-length fields, use the observed mean encoded length, not the block length.

Required Buffer by Deployment

BDP defines the minimal buffer size required to achieve constant TPS at a given RTT. The numbers below assume a 10 Gbps link.

Deployment	RTT	Required Buffer (10Gbps)
CPG	~5 µs	~6 KB
Same AZ	~100 µs	~125 KB
Cross AZ	~1 ms	~1.25 MB
Cross Region	~50 ms	~62.5 MB

Key insight

The same Aeron configuration that works perfectly in same-AZ testing can fail catastrophically cross-AZ or cross-region. Buffer sizes must scale with RTT. Many production issues trace back to this mismatch. Same-AZ (100 µs RTT) needs tiny buffers; cross-region (tens of ms RTT) needs much larger ones.