Hardening a STARK Proving Service: Real Benchmarks, Not Guesses
A public "push a program, get a proof" endpoint has an obvious problem: proving is CPU-intensive, and nothing was stopping a small request from asking for a lot of it. Here's what got measured before picking a number.
An external review of this repo correctly flagged that /v1/proofs had no request body limit, no program-size cap, no timeout, and no concurrency limit — a small HTTP request could ask the server to prove an arbitrarily large program, tying up CPU and memory with no backstop. That's a real gap. Fixing it properly meant not guessing at a number.
The first number was wrong
The obvious approach: cap the number of instructions a submitted program can have. A first pass picked 10,000 as "generous but bounded" — reasonable-sounding, untested. Writing a test for it (prove a program at exactly the limit) exposed the problem immediately: the test didn't finish in any reasonable time.
So we measured. zkvm prove, release build, on ordinary dev-laptop hardware, programs shaped like INIT 0 followed by N ADD 1 instructions:
| Instructions | Prove time |
|---|---|
| 1,000 | 1.2s |
| 2,000 | 6.0s |
| 4,000 | 32.0s |
That's clearly worse than linear — doubling the program length roughly quintupled proving time in this range, consistent with the FFT-heavy operations STARK proving involves. At that growth rate, 10,000 instructions would have taken minutes, not seconds — burning full CPU the whole time before any timeout could even help, which would have made the "limit" actively counterproductive: a request within the allowed size that still exhausts the server just more slowly.
The number that's actually in the code now
MAX_PROGRAM_INSTRUCTIONS is 2,000 — chosen to keep worst-case proving time around 6 seconds on the hardware it was measured on, an order of magnitude under the request timeout, with real margin for slower or loaded production hardware. The comment in the source says exactly this, with the real numbers, not a rationale invented after the fact:
/// Tuned against real measurements on ordinary dev-laptop hardware
/// (`zkvm prove`, `INIT 0` + N `ADD 1` instructions, release build):
/// 2,000 instructions ~6s, 4,000 instructions ~32s -- proving time here
/// is clearly worse than linear in program length ... Re-measure before
/// raising this -- it is load-bearing for REQUEST_TIMEOUT actually being
/// a backstop rather than routine behavior.
pub const MAX_PROGRAM_INSTRUCTIONS: usize = 2_000;The rest of the hardening
Three more limits, applied uniformly to every route:
- Request body cap (1 MiB) — via axum's built-in
DefaultBodyLimit, no extra dependency needed - Per-request timeout (60s) —
tower_http::timeout::TimeoutLayer, which returns a proper 408 response rather than the awkward error-type wrappingtower's own core timeout needs for an axum service - Concurrency cap (4 in-flight requests) —
tower::limit::ConcurrencyLimitLayer, so parallel proving requests queue instead of running unbounded in parallel
The instruction-count check itself lives in one place (check_program_size) and is called at every entry point that can trigger proving or verification work — the plain /v1/proofs//v1/verify handlers, the /v1/backends/* router path, and the MCP prove/verify tools — rather than trusting each call site to remember to add it separately.
What's still not covered: auth/API keys, per-tenant accounting, structured audit logs, and metrics/tracing don't exist yet. A queued request that never gets served still doesn't turn into an explicit 503 today. These are named gaps, not silent ones — see docs/THREAT_MODEL.md.