Introduction

llama-dash is the front door for a self-hosted inference box. It gives a small team one place to load models, create client keys, inspect requests, set routing rules, and watch the host. The current backend is llama-swap, which runs llama.cpp model servers.

Clients call llama-dash, not llama-swap. llama-dash handles proxy policy, logging, authentication, and routing. The backend starts model processes and runs inference.

Start here

• Have an NVIDIA or AMD inference host and want the bundled stack? Start with Docker Compose.
• New to llama.cpp and llama-swap? Read llama.cpp & llama-swap to see how much setup each install path needs.
• Already run llama-swap? Use Manual setup for a production install, or Contributing for development.
• Have a running dashboard and want to send a request? Read Connect a client.

What you get

• Dashboard pages for model state, request history, host health, and raw logs.
• An OpenAI- and Anthropic-compatible proxy at /v1/*.
• API keys, rate limits, model restrictions, aliases, and ordered routing rules.
• Optional direct-upstream routing and encrypted credentials for provider APIs and MCP relays.
• Prometheus metrics at /metrics.

Learn more