How it works
Architecture
One public port, three surfaces, and a clear split of responsibilities.
llama-dash sits in front of an inference backend. It exposes one public port with three paths:
client ──► llama-dash :3000 ──► inference backend ──► model server procs
│
├─ UI (/) the dashboard
├─ Admin (/api/*) data for the UI
└─ Proxy (/v1/*) what clients actually hitIn the bundled Compose setup, llama-swap is private to the Compose network. Clients and operators use llama-dash on port 3000.
Responsibilities
| llama-dash owns | The inference backend owns |
|---|---|
| Proxy policy, routing, and filters | Local model processes |
| Authentication and API keys | Loading/unloading models |
| Request logging and metrics | Running inference |
| Backend normalization | GGUF / llama.cpp specifics |
llama-dash does not run inference. The selected backend does that job, currently llama-swap over llama.cpp. It is designed for one team operating one machine, not as a multi-tenant service.
The three surfaces
- Proxy (
/v1/*): OpenAI- and Anthropic-compatible requests from clients. See The proxy. - Admin API (
/api/*): JSON used by the dashboard. Dashboard authentication protects it. See the admin API reference. - UI (
/): The operator dashboard. See The dashboard.
Last updated on