MTE Relay Performance
MTE Relay is a drop-in encrypted HTTP gateway. It is the same engine behind both MTE Relay Server and MTE API Relay, so the performance characteristics on this page apply to either deployment. An application sends ordinary HTTP to an outbound relay; the relay MTE-encrypts the entire request; a peer relay in front of the origin decrypts it and forwards plaintext to the origin; and the response makes the same trip in reverse. Every byte of every request and response is encrypted in transit between the two relays, with no changes to the client or the origin service.
This page has two goals: to show that MTE Relay is highly performant, and to help you choose the right deployment size for your traffic.
Performance at a glance
A pair of standard 4 CPU / 4 GB relay instances moves about 9,150 encrypted requests per second of 10 KB JSON, roughly 92 MB/s (0.73 Gbit/s) of fully encrypted application payload, or about 790 million requests per day, with a measured error rate of zero. Across the four runs of this configuration, 12,571,633 requests completed with no errors, and the load generator compared every response body with the request it sent. The whole test run moved 131 million requests through the relays without one error.
The price of the encryption is small and predictable:
- At light load, the full encrypt → transmit → decrypt → origin → encrypt → transmit → decrypt round trip adds about 1 millisecond per request (1.12 ms through the relay pair against 0.15 ms talking to the origin directly). For interactive APIs this is imperceptible.
- Past its ceiling the gateway saturates gracefully, never destructively. When offered more work than it can encrypt, requests queue: latency rises linearly and predictably, throughput holds flat, and nothing fails. At 8× saturating demand, 99% of requests still completed in under 34 ms. A separate run at 1,000 connections held 8,337 req/s with no errors.
Throughput scales with CPU, and per core it is highest on small instances: about 2,760 requests per second per relay core at 1 CPU, 2,600 at 2 CPU, 2,290 at 4 CPU, and 1,700 at 8 CPU, where the test rig itself becomes the limit. Memory is not a throughput factor: it tracks payload size, not demand.
How MTE Relay works
Because the client and origin are untouched, MTE Relay drops into an existing architecture as an encrypted hop. The only resource it consumes is CPU to encrypt and decrypt; everything else on this page follows from that.
How we measured it
Load was generated with k6, the open-source load-testing tool from Grafana Labs. k6 reports latency percentiles (p90, p95, p99) directly in its summary output, so every latency figure on this page is taken straight from the tool rather than computed by us.
The test uses maximum pressure connections, not realistic users. Each connection is a k6 virtual user (VU) running closed loop with zero think time: it fires its next request the instant the previous response arrives. A single such connection is itself a heavy load source: one connection alone extracts about 1,000 requests per second from the gateway. These connections are best thought of as units of offered pressure, each equivalent to hundreds of real API clients.
| Test connections | Each connection was sending | Total demand delivered |
|---|---|---|
| 8 | 840 req/s | 6,717 req/s |
| 16 | 558 req/s | 8,924 req/s |
| 32 | 289 req/s | 9,262 req/s |
| 64 | 146 req/s | 9,333 req/s |
| 128 | 72 req/s | 9,198 req/s |
| 256 | 35 req/s | 8,978 req/s |
The gateway reaches its ceiling of about 8,900 to 9,300 req/s once total demand exceeds it (about 16 maximum pressure connections); beyond that point the same total throughput is shared across more connections. The sweep then pushed to 256 simultaneous saturating connections, roughly 16× the demand needed to saturate the gateway, to prove overload safety, and a separate run went to 1,000. The error count stayed at zero throughout.
Measured throughput and latency
Workload: a POST of ~10 KB JSON, sustained 60 second holds per load level, with a
co-located k6 load generator over plain HTTP (no TLS) to isolate the
cost of MTE encryption. Each value is the median of four runs; worst case is the
median of the four runs' slowest request.
| Offered load (vs capacity) | Throughput (req/s) | Avg latency | p90 latency | p99 latency | Worst case | Errors |
|---|---|---|---|---|---|---|
| Light (~70%) | 6,717 | 1.1 ms | 1.4 ms | 1.8 ms | 3 ms | 0 |
| Saturating (100%) | 8,924 | 1.7 ms | 2.5 ms | 3.5 ms | 6 ms | 0 |
| Overload (~2×) | 9,262 | 3.4 ms | 5.2 ms | 7.4 ms | 16 ms | 0 |
| Overload (~4×) | 9,333 | 6.8 ms | 10.9 ms | 16.1 ms | 80 ms | 0 |
| Overload (~8×) | 9,198 | 13.8 ms | 22.8 ms | 33.8 ms | 148 ms | 0 |
| Overload (~16×) | 8,978 | 28.4 ms | 46.2 ms | 67.7 ms | 172 ms | 0 |
Across the four runs of this configuration: 12.6 million requests, 0 failures, 0 mismatched responses.
Throughput holds flat at capacity no matter how hard the gateway is pushed:
Under overload, the only thing that grows is queueing delay, and it grows linearly, with no cliffs and no error storms:
What this means in real terms
9,150 encrypted requests/second is, equivalently:
| Client profile | Request rate per client | Clients supported by one relay pair |
|---|---|---|
| Mobile/web user (6 requests per minute) | 0.1 req/s | ~91,000 concurrent users |
| Active interactive session (20 requests per minute) | 0.33 req/s | ~27,000 concurrent sessions |
| Busy machine to machine integration | 10 req/s | ~900 concurrent integrations |
| Maximum pressure benchmark connection | 35 to 1,000 req/s | ~16 (the saturation point above) |
- For an interactive API: the relay adds about one millisecond. A user cannot perceive it.
- For a high volume service: one standard relay pair carries ~790 million encrypted API calls per day at 10 KB each.
- For reliability engineering: overload manifests as added queueing delay, not as errors. At 16× saturating demand, and at 1,000 connections in a separate run, the gateway did not drop, corrupt, or reject a single request.
Real-world example: across the public internet
This sweep was measured on v4 in June 2026; v5 has not been measured across a WAN yet.
Every figure above was measured with the load generator and both relays inside a
single cluster. That isolates the cost of MTE encryption cleanly, but it says nothing
about how the relay feels on a real network. To close that gap we ran the same A/B
sweep across an actual WAN: the client and outbound relay on a workstation over a
residential internet connection, and the inbound relay and origin on AWS EKS in
us-east-1, each behind an Elastic Load Balancer.
Both arms cross the WAN exactly once — the control arm goes straight to the origin in plaintext, the test arm goes through the MTE relay pair — so the difference between them isolates the relay, not the network. Each side ran a single 2 CPU / 2 GB relay with in-memory state.
| Metric | Result |
|---|---|
| Median latency added per request | +2.07 ms (mean of 9 unsaturated cells) |
| As a fraction of the WAN round trip | +3.7% (against the 56 ms baseline RTT) |
| Throughput cost at equal offered load | −3.14% |
| Failed requests | 2 of 1,008,891 (0.0002%) |
| Response integrity | 100.00% byte-intact, both arms |
The structural finding is the important one: the relay's cost is a small, roughly fixed amount per request — not a multiplier on network distance. The relay performs the same encode/decode work regardless of how far the packet travels; what changes between environments is the denominator. An in-cluster A/B on the same cloud infrastructure the same day measured +2.65 ms; across the WAN the same work measured +2.07 ms — the same magnitude in two very different environments. On a LAN, a couple of milliseconds can multiply a sub-millisecond round trip several times over. Across a real internet path it disappears into a few percent of a 56 ms one.
- The further traffic travels, the cheaper encryption looks. Because the overhead is fixed, its relative cost falls as round-trip time rises — exactly the situation of any API served across regions, or to mobile clients on cellular networks.
- Saturation came from the network, not the relay. In the cells that plateaued, the residential uplink (~40 Mbit/s each way) was the ceiling — and it capped the plaintext control arm just as hard. Both arms degraded into longer queues, never into errors.
- Integrity held throughout. Every response in both arms was byte-intact, with no MTE protocol errors at any load level.
The control arm is plaintext HTTP, so these figures are the relay's cost against a bare, unencrypted origin. Against a TLS-terminated baseline the gap would be smaller — TLS is not free either, and in this test MTE is carrying the encryption that TLS would otherwise be providing.
Resource footprint
Every instance shape measured ran with zero errors through full overload sweeps:
| Per relay instance | Memory at full load (10 KB payloads) | Saturation (10 KB) |
|---|---|---|
| 4 CPU / 4 GB (standard) | 92 to 144 MiB | ~9,150 req/s |
| 2 CPU / 2 GB (small) | 100 to 152 MiB | ~5,200 req/s |
| 1 CPU / 1 GB | 86 to 116 MiB | ~2,760 req/s |
| 8 CPU / 8 GB | 89 to 121 MiB | ~13,570 req/s |
CPU is the limiter through 4 CPU per relay; at 8 CPU the test rig's own ceiling is, with relay CPU to spare. Memory does not grow with offered load. It tracks payload size: 42 to 91 MiB per relay at 2 KB, 92 to 152 MiB at 10 KB, and 250 to 411 MiB at 100 KB, against about 2 GB per relay for v4 at 10 KB. Under overload the gateway degrades in latency only.
Sizing by hardware: cores → throughput
The gateway was measured at four instance sizes with the identical build and workload (10 KB payloads), giving a measured cores → throughput curve:
| Per relay CPU | Sustained ceiling (req/s) | Per relay core |
|---|---|---|
| 1 (measured, one run) | ~2,760 | ~2,760 req/s |
| 2 (measured) | ~5,200 | ~2,600 req/s |
| 4 (measured) | ~9,150 | ~2,290 req/s |
| 8 (measured) | ~13,570 | ~1,700 req/s |
Throughput rises with every core, though not in a straight line: ×1.88 from 1 to 2 CPU, ×1.76 from 2 to 4, and ×1.48 from 4 to 8. The 8 CPU point sits at a ceiling of about 14,000 req/s that the test rig also imposed on the two 4 CPU relay cells, with the relays using about half their cores, so read it as a floor.
Practical guidance: size by CPU. Small instances deliver the most per core, but splitting the same cores across replicas costs throughput, because the inbound relays share state through Redis: one 4 CPU relay (9,152 req/s) moved more than two 2 CPU relays with Redis (8,039 req/s). Add replicas for high availability and size each one for throughput. Confirm sizes beyond the measured 1 to 8 CPU range by measurement before relying on them for production.
On memory: v5 stayed under 160 MiB per relay at 10 KB on every shape and near 410 MiB at 100 KB, so the 1, 2, 4 and 8 GB limits used in these runs leave ample headroom.
Payload-size sensitivity
Three full overload sweeps at 2 KB, 5 KB, and 10 KB bodies on the 2 CPU / 2 GB shape, four runs each, 0.000% errors in every run, give a simple, accurate cost model:
| Payload | Saturation (req/s) | Payload moved | Light-load avg latency | p90 latency | p99 latency |
|---|---|---|---|---|---|
| 2 KB | 6,674 | ~13 MB/s | 1.2 ms | 32.4 ms | 44.8 ms |
| 5 KB | 6,003 | ~30 MB/s | 1.4 ms | 33.3 ms | 45.3 ms |
| 10 KB | 5,307 | ~53 MB/s | 1.5 ms | 36.9 ms | 49.6 ms |
The p90 and p99 columns are measured under heavy overload (128 maximum pressure k6 connections); the average latency is at light load (8 connections). The three points fit a simple linear cost model closely:
Relay pair capacity cost per request ≈ 141 µs fixed + 4.8 µs per KB (within 1 percent of all three measurements; v4 was 161 µs + 9.8 µs per KB)
What this means in practice:
- Per request overhead dominates. Even at 10 KB, 75% of the cost is payload independent. Shrinking payloads 5× raises request throughput only about 1.26×, so for small payload APIs, plan capacity by request rate, not by bandwidth.
- Larger payloads are relatively cheaper to encrypt: moving the same data in 10 KB requests costs about 4× less gateway capacity than moving it in 2 KB requests.
- Outside this sweep, the 4 CPU shape sustained 11,729 req/s at 1 KB and 2,757 req/s at 100 KB (one run each), about 276 MB/s of payload.
- To estimate saturation for any payload size and shape: take
1 ÷ (141 µs + 4.8 µs × KB)and scale by the CPU curve above.
High availability and horizontal scaling
For production resilience the gateway runs multiple relay instances behind a load
balancer. In v5 only the inbound relays share encryption state, through one
Redis; outbound relays keep their pairs in process. Any inbound
instance can then serve any client's traffic, an instance can be replaced without
breaking sessions, and capacity grows by adding instances. These runs used plain
Kubernetes Services (L4 load balancing) with LOCAL_STATE_CACHE off.
At the 2 CPU shape, the second instance adds about 1.5× and the third about 1.3×, at 5 KB and 10 KB alike, with 0.000% errors at every point:
| Deployment | 5 KB | 10 KB |
|---|---|---|
| 1 instance × 2 CPU, no Redis | 5,854 | 5,198 |
| 2 instances × 2 CPU, one Redis | 8,953 | 8,039 |
| 3 instances × 2 CPU, one Redis (one run) | 11,563 | 10,116 |
| 2 instances × 4 CPU, one Redis | 13,391 | 11,776 |
- One Redis instead of two. v4 also needed a shared store for the outbound relays; v5 keeps outbound pairs in process, so a highly available deployment runs one Redis.
- Replicas cost some throughput. One 4 CPU relay (9,152 req/s) moved more than two 2 CPU relays with Redis (8,039 req/s) at 10 KB, so add replicas for availability and size each one for throughput.
- Relay memory stays low: 62 to 124 MiB per relay with two instances.
Reading these numbers. As everywhere on this page, the co-located load generator makes absolute RPS rig relative. The portable result is the shape: throughput rises with both replicas and cores, at 0.000% error.
Deployment tiers: Small, Medium, Large, Enterprise
The measured results reduce to four standard deployment tiers. Two planning rules are baked into the recommendations:
- Run at ≤60% of measured saturation. The gateway is provably safe far beyond that (zero errors at 16× overload and at 1,000 connections), but at ≤60% utilization p99 latency stays in the single digit to low tens of milliseconds.
- Capacities below are quoted for ~10 KB payloads, the most demanding case measured. Smaller payloads raise request capacity (×1.13 at 5 KB, ×1.26 at 2 KB); adjust before picking a tier.
| Small | Medium | Large | Enterprise | |
|---|---|---|---|---|
| Relay deployment | 1 × 2 CPU / 2 GB, self-contained state | 2 × 2 CPU / 2 GB + one Redis (2 CPU / 1 GB) for the inbound relays | 2 × 4 CPU / 4 GB + one Redis (2 CPU / 1 GB) for the inbound relays | N independent "cells" of the Large tier |
| Total hardware (both ends) | 4 CPU / 4 GB | 10 CPU / 9 GB | 18 CPU / 17 GB | 18 CPU / 17 GB per cell |
| Measured saturation | 5,307 req/s | 8,319 req/s | 13,063 req/s | ~13,000 req/s per cell |
| Recommended sustained load | ≤ 3,100 req/s | ≤ 4,900 req/s | ≤ 7,800 req/s | ≤ 7,800 req/s × N |
| Requests per day | ≤ 270 M | ≤ 420 M | ≤ 670 M | 670 M × N |
| Encrypted data per day (10 KB) | ≤ ~2.7 TB | ≤ ~4.2 TB | ≤ ~6.7 TB | ~6.7 TB × N |
| Concurrent users (1 req/min) | ~190,000 | ~290,000 | ~470,000 | ~470,000 × N |
| High availability | No (single instance; clients pair again after a restart) | Yes | Yes | Yes, plus cell-level isolation |
| Typical fit | Departmental APIs, internal services, pilots | Consumer app or B2B platform | High-volume consumer platform | National-scale / multi-tenant / multi-region |
Choosing a tier from your numbers:
- Take your peak sustained request rate (not the daily average; size for the busiest hour).
- Adjust for payload:
capacity ≈ tier rating × (141 + 4.8×10) ÷ (141 + 4.8×KB). - Pick the smallest tier whose recommended load covers it; step up one tier if you need high availability (Medium is the smallest highly available tier).
Worked example: a consumer API peaking at 3,000 req/s with ~5 KB payloads. The 5 KB capacity factor is ×1.13, so Small's effective rating is about 3,500 req/s and Medium's about 5,500 req/s. Small fits without high availability; Medium fits with headroom and provides HA.
Enterprise scaling notes:
- Grow by adding cells (an independent Large tier gateway pair with its own Redis), splitting traffic by client population, region, or DNS. Cells share nothing, so capacity grows linearly and a cell failure is contained.
- Within a cell, three 2 CPU relays with one Redis sustained 10,116 req/s at 10 KB with no errors (one run); certify larger cells before relying on them.
Test conditions and scope
- Topology: client → outbound relay → encrypted hop → inbound relay → origin,
with a 1,024 pair pool created at startup. v5's defaults grow the pool from 25 to
1,024 pairs on demand (5.0.0 shipped with a ceiling of 100), so set
CLIENT_MAX_PAIRSto your peak concurrency per outbound host andMAX_PAIRS_PER_CLIENTon the receiving relay to at least that. Headline numbers are a single 4 CPU / 4 GB instance at 10 KB with in-process state; the sizing curve adds measured 1, 2 and 8 CPU points; the payload model was measured at the 2 CPU / 2 GB shape with 2, 5 and 10 KB bodies; the multi-instance results use two and three instances behind plain Kubernetes Services, with one Redis for the inbound relays. Each configuration ran four times and the page reports the median; the 1 CPU, three instance, 1 KB, 100 KB and 1,000 connection figures are single runs. - Tooling: load was generated and measured with k6 0.54.0 (Grafana Labs); the highly available deployments coordinate relay state through Redis; everything ran on Kubernetes. All latency percentiles (p90, p95, p99) are reported directly by k6.
- Environment: single node Kubernetes with 30 CPUs and the k6 load generator co-located in the cluster, so the absolute numbers are conservative. A direct to origin reference on the same rig peaked near 41,000 req/s. The 8 CPU and the two 4 CPU relay results sit near a shared ceiling of about 14,000 req/s with relay CPU to spare, so treat them as floors.
- Protection settings: MKE with Kyber 512, to match the v4 runs. v5's default, Kyber 1024, measured the same throughput (9,190 against 9,152 req/s at 4 CPU, one run). Response coalescing ran at its 2 ms default; turning it off cost about 9%.
- Real-world WAN validation (v4, June 2026): a separate A/B sweep placed the
client and outbound relay on a residential internet connection and the inbound
relay and origin on AWS EKS in
us-east-1(one 2 CPU / 2 GB relay, in-memory state, one load balancer hop per arm). Both arms crossed the WAN exactly once, which isolates the relay's cost from the network's: 1,008,891 requests, 2 failures, 100% response integrity. Absolute throughput there is specific to that internet connection; the delta between arms is the portable result. - Isolation: all traffic was plain HTTP by design, so the figures isolate MTE encryption cost. TLS termination would add its usual, separate cost.
Bottom line: at the standard 4 CPU / 4 GB shape, MTE Relay v5 delivers fully
encrypted HTTP at about 9,150 requests per second per relay pair, enough for roughly
91,000 typical concurrent users, with about 1 ms of added latency at light load,
every response verified, and zero errors from light load to 1,000 connections.
Capacity follows CPU on a measured curve, payload size on a measured cost model
(~141 µs + ~4.8 µs/KB per request), and instance count on measured scale-out data.
Pick Small (≤270 M req/day), Medium (≤420 M/day, HA), Large (≤670 M/day,
HA), or Enterprise (linear growth by cells) from your peak traffic and payload
size, then confirm with a one-hour certification sweep.