The proxy

The proxy is the /v1/* pass-through that clients connect to. It forwards OpenAI and Anthropic calls to the selected backend (or a direct upstream) while logging every exchange.

Streaming and token counting

• Streaming is preserved. SSE chunks are written to the client as they arrive — never buffered and flushed at the end.
• Token counts are scraped in flight. A tee'd scan reads the final usage chunk (OpenAI flat prompt_tokens/completion_tokens, or Anthropic nested input_tokens/output_tokens) without consuming the body before forwarding.
• One row per completed request. A requests row represents a finished exchange, not an attempt. If the client disconnects mid-stream, the row is still enqueued with whatever usage was captured.

OpenAI and Anthropic compatibility

The proxy understands both request shapes:

• OpenAI — /v1/chat/completions, flat usage.
• Anthropic — /v1/messages, /v1/messages/count_tokens, nested message.usage, message_stop stream terminator.

Cost estimates

Request cost is estimated from upstream usage counters using a startup-cached models.dev pricing catalog. Unknown or unavailable pricing yields a null cost rather than a guessed value.

Transform pipeline

POST /v1/* requests pass through an ordered transform pipeline between auth and forwarding. The body is parsed once when needed and re-serialized only if mutated:

1. API key allow-list check
2. Routing rule evaluation
3. Model alias resolution
4. System-prompt injection
5. Request size limits

Routing rules are ordered and first-match-wins. See Request logging for what is captured afterwards.