Measuring session-start latency, and what we found when we did

Our README made a claim about the warm pool: session start is a claim against an already-running pod, not a cold boot, and it should beat GKE's own managed cold-start equivalent. That sentence had zero backing in the repo — no script, no numbers, no methodology. We wanted a real measurement before we kept saying it.

We wrote scripts/benchmark_warm_pool.py against our real hosted control-plane deployment on Cloud Run (us-central1), using a fresh, disposable account created just for the benchmark. It measures two series, five samples each, every session destroyed immediately after its latency sample: claim_latency_ms (destroy, then immediately create the next session — the best realistic chance for a warm-pool claim to land) and cold_start_latency_ms (a 15-second sleep between destroy and create, to reduce the chance of grabbing an immediately-recycled pod). Latency is measured client-side around POST /v1/sandboxes, and a 201 there is already a usable sandbox — the control-plane blocks on the sidecar's /configure call succeeding first.

The two code paths we were actually comparing

Reading the code told us what the two series really were. Every SandboxManager.create_session tries the claim path first and only falls back to a full cold build if no warm-labeled pod is found — but both paths end at the same blocking /configure gate before a 201 is returned:

The first run, and why the two series matched

The first run, on 2026-07-10:

The two series didn't differ in any way attributable to a warm pool — and reading the code told us why. SandboxManager.create_session always tries _claim_warm_pod_via_k8s first, but nothing in the control-plane's lifespan() ever constructed a WarmPoolManager, and its get_warm_pool() singleton had zero callers anywhere in the codebase outside its own definition. Every request was silently taking the cold-create path — both series were the same code path measured twice.

We fixed the wiring, then re-measured

So we fixed the actual gap: lifespan() now calls get_warm_pool() (guarded to RUNTIME_MODE=k8s), we rebuilt and redeployed the control-plane with WARM_POOL_SIZE=1set, and confirmed it live. Along the way we found something the first investigation didn't know: SandboxManager already recycles destroyed pods into the pool=warm label state on every destroy, independent of WarmPoolManager entirely — which is why five warm-labeled pods already existed in the cluster before this deploy. A direct curl confirmed the claim path really does pick up an already-recycled pod.

We re-ran the exact same benchmark against the redeployed service:

The claim series was actually about 140ms slower, median to median. We're reporting that plainly because it's the honest result, not the one we expected. Our best guess, not yet isolated: the Kubernetes API list-and-patch round trip _claim_warm_pod_via_k8sdoes, plus its pod-health check, may cost as much or more than this small, already-warm cluster's actual cold create_namespaced_podcall — a real production-scale cluster would likely pay a much higher node-provisioning and image-pull cost on the cold path that this dev-scale cluster doesn't.

We never had a comparable GKE Agent Sandbox number to compare against in the first place. If speed is still worth pursuing, the next step is profiling where each path actually spends its time — not assuming the wiring fix alone was sufficient.