The KeyForge pattern: virtual keys, real keys, and the audit chain in between

The problem: three problems at once

If you're running a marketplace where multiple agents share underlying model APIs, you're eating three problems simultaneously:

1. Rate limits. OpenAI's 500 RPM on a Tier 4 org is a real constraint when 30 agents fan out.
2. Attribution. Whose spend is that? Whose bill? Which agent is running away with the token budget?
3. Security. Do you really want an agent's workflow file to contain your production sk-... key?

The pattern

KeyForge introduced a pattern that solves all three: virtual keys that look like real keys, backed by a proxy that swaps them for real ones just-in-time.

An agent's workflow references vk_prod_openai_gpt4o. The gateway:

• Authenticates the vk_ token (sha256 hash lookup).
• Checks the scope (allowed models, RPM, daily quota, spend cap, expiry).
• Selects a real key from the linked pool (skipping any blacklisted for the current cool-off window).
• Signs the request, tracks tokens/cost, writes a chain-linked audit entry.
• Returns an OpenAI-compatible response envelope.

Neither the agent nor the workflow file ever sees the real key.

Rate-limit resilience

The dirty secret of production LLM apps is that provider rate limits burst before they stabilize. A well-tuned virtual key pool with 3–5 backing real keys can cut 429s to under 1%. When a real key trips a burst limit, the gateway marks it blacklisted for 60s and continues serving the request from the next key in the pool.

Cost accounting that actually works

Every audit entry carries tokens_prompt, tokens_completion, provider, model, and a cost_usd_est computed from a maintained price table. Aggregate by virtual key for "how much did agent X cost this week," or by job_id for "how much did we spend delivering this to this employer." The employer never sees the internals — but internally, unit economics are exact.

Rotation without downtime

To rotate a real key, add the new one to the linked pool, wait until traffic organically shifts, then remove the old. No agent files are edited. No secrets are deployed. This is table stakes at scale.

Concurrency: don't fork the chain

This one bit us in v1. Two concurrent gateway calls each read the current chain head, compute their HMAC against the same prev_hmac, and both insert. Now verify_chain walks the log by timestamp ascending and detects the fork.

Fix: an asyncio.Lock per mode inside the gateway process serializes the append. For horizontally scaled deployments, use an atomic MongoDB findOneAndUpdate on a chain-head document, or a unique index on prev_hmac with retry on DuplicateKeyError.

Beyond the marketplace

Even if you're not building a marketplace, the virtual-key + gateway pattern is worth stealing. It's the difference between "we have logs somewhere" and "we can prove exactly what our production LLMs did, in order, forever."