AWS Hard Limits You Cannot Tune Past

Before you touch a single Aeron parameter, know the wall you’re up against. AWS imposes five hard limits on EC2 networking. These are physics and infrastructure constraints — no amount of Aeron tuning can overcome them.

The lesson is simple: verify you’re not hitting any of these ceilings first. Tuning a system that’s already pinned against a PPS cap or running at 90% utilization just rearranges the symptoms.

For how Aeron’s transport actually moves bytes across the wire, defer to The Aeron Files. This page stays focused on the AWS ceilings around it.

1. PPS (Packets Per Second)

Every EC2 instance has a packet-rate ceiling. Different instance types and sizes have different PPS limits, and you can’t exceed the instance’s PPS ceiling regardless of tuning.

PPS bites small-packet, high-message-rate workloads first — exactly the profile of low-latency trading traffic. If your throughput plateaus while bandwidth utilization stays low, you’re packet-bound, not bandwidth-bound. Scaling up the instance size (or batching messages into fewer, larger packets) is the only real fix.

2. Per-Flow Limit

A single flow — one 5-tuple — is capped:

• 5 Gbps within a VPC (standard)
• 10 Gbps within a CPG (Cluster Placement Group)

You may need multiple flows to saturate the available bandwidth. One Aeron stream pinned to a single 5-tuple cannot exceed the per-flow cap no matter how wide the underlying link is.

Key insight

If aggregate throughput stalls at 5 Gbps inside a VPC, suspect the per-flow limit before you suspect Aeron. Spreading traffic across multiple flows lifts the ceiling to the instance bandwidth — the single-flow knob does not.

3. Bandwidth

PPS and bandwidth are independent limits. You can hit either one first.

Packet Per Second × Packet Size ≤ Bandwidth

Read the formula both ways. At small packet sizes, PPS is the binding constraint and you’ll never approach the bandwidth number. At large packet sizes, bandwidth binds first and PPS sits idle. Plot your real packet-size distribution against both ceilings to find which one you’ll hit.

4. Utilization-Latency Relationship

Unlike the PPS and bandwidth caps above, this isn’t a hard physical wall — it’s an operating target. For a latency-sensitive single-flow / single-core hot path, keep bandwidth and CPU ≤ 60% to leave headroom for jitter and micro-bursts. That figure sits inside the broader 50–70% queuing-knee band; ≤ 60% is the conservative end of it. For why this band exists — the 1 / (1 − ρ) response-time curve, the ~70% knee, and the cases that raise the ceiling (multi-core fleets, batched service) — see Transport Performance Factors.

The reason to be conservative on a hot path: averages hide bursts. A 5-minute average that “never exceeds 80%” can still mask cores pinned at 100% for seconds — and it’s those sub-interval peaks, not the average, that queue and blow out p99. Steady-state numbers lie: 90% utilization can look fine in p50 yet produce terrible p99 under any burst. The headroom isn’t waste; it’s what absorbs the micro-bursts.

Key insight

Treat ≤ 60% as a conservative p99 budget for a single hot-path resource, not a universal law. Multi-core / multi-server pools can safely run hotter (80–90%), and Aeron’s smart batching raises the ceiling by removing arrival randomness. But on a pinned, single-flow hot path, ≤ 60% is the rule most often violated — and the one that most often explains a mysterious p99.

5. BDP (Bandwidth Delay Product)

BDP determines the minimum buffer size required for a given throughput at a given RTT.

Bytes in Flight = Bandwidth × Latency
  0.125 Mbit = 10 Gbit/s × 100 µs
  1.25  Mbit = 10 Gbit/s × 1 ms

Undersize the buffer below the BDP and the sender stalls waiting for ACKs, capping throughput. Buffer sizing scales with RTT, so a configuration that’s correct same-AZ can starve cross-AZ or cross-region.

Go deeper

This page only states the BDP ceiling. For the full sizing methodology — the 2x–4x rule of thumb, per-deployment buffer tables (CPG, same-AZ, cross-AZ, cross-region), and the throughput/tail trade-offs — see the Bandwidth Delay Product reference.

How to use these limits

Treat the five as a pre-flight checklist. Walk it before reaching for any Aeron knob:

1. Am I PPS-bound? Throughput flat, bandwidth low → packet-rate ceiling.
2. Am I flow-bound? Stuck at 5 Gbps in a VPC (or 10 Gbps in a CPG) → spread across more flows.
3. Am I bandwidth-bound? PPS × packet size approaching the link rate → larger instance.
4. Is a hot-path resource over ~60% utilization? Then its p99 is at the mercy of the next micro-burst.
5. Are my buffers below BDP? Then throughput is capped by RTT, not by Aeron.

Clear all five, then start tuning. For the protocol-level mechanics underneath these ceilings, continue to The Aeron Files.