ld
Getting started

Quick start

Run llama-dash and llama-swap together with Docker Compose.

The fastest way to run llama-dash is with Docker Compose. Both setups pull the published ghcr.io/ndom91/llama-dash image, bundle it with llama-swap, use ./config/config.yaml for the llama-swap config, mount ./models/ for model files, and expose llama-dash on http://localhost:3000.

Before you start

  • Install Docker Engine with the Docker Compose plugin.
  • For NVIDIA, install the NVIDIA Container Toolkit. For AMD, use a Linux host with a working Vulkan driver and the GPU available at /dev/dri.
  • Download a GGUF model file. Its path must match the cmd you place in config.yaml.

You don't need to install llama.cpp or llama-swap: the Compose file runs llama-swap's recommended unified image, which includes llama-server. See llama.cpp & llama-swap for what that image provides.

The bundled Compose files target NVIDIA or AMD GPUs. If you already run llama-swap elsewhere or need a non-GPU setup, use Manual setup instead.

Get the Compose files

The application image is prebuilt, but the repository provides the Compose files and example llama-swap configuration.

git clone https://github.com/ndom91/llama-dash.git
cd llama-dash

Configure the environment

cp .env.example .env

Set BETTER_AUTH_SECRET to the output of openssl rand -base64 33. The compose files load .env via env_file and won't start without it. Leave BETTER_AUTH_URL empty for a local install; set it to your browser-visible HTTPS origin when deploying behind a reverse proxy or using passkeys remotely. CREDENTIAL_ENCRYPTION_KEY (32+ characters) is optional until you store an upstream provider credential. See Configuration for details.

Create the llama-swap config

config/config.yaml is llama-swap's config file, not a llama-dash config. It lists the models llama-swap can start. The dashboard's Config page edits this same file.

cp config/config.example.yaml config/config.yaml

Place your GGUF file in ./models/, then set a model ID and its matching container path in config/config.yaml. For example, a file at ./models/my-model.gguf needs /models/my-model.gguf in the command:

models:
  my-model:
    cmd: llama-server --port ${PORT} --model /models/my-model.gguf

my-model is the model ID clients use. See llama-swap's configuration overview for more options, and the config file reference for how llama-dash reads and writes the file.

Start the stack

Pick the compose file that matches your GPU vendor.

docker compose -f docker-compose.amd.yaml pull
docker compose -f docker-compose.amd.yaml up -d

Runs ghcr.io/mostlygeek/llama-swap:unified-vulkan, passes through /dev/dri, and also mounts /dev/dri into llama-dash so AMD GPU stats work in the dashboard.

docker compose -f docker-compose.nvidia.yaml pull
docker compose -f docker-compose.nvidia.yaml up -d

Runs ghcr.io/mostlygeek/llama-swap:unified-cuda13 and requests gpus: all for the llama-swap service. This requires the NVIDIA Container Toolkit on the host.

unified-cuda13 supports Turing (RTX 20xx, GTX 16xx) and newer GPUs. For Pascal cards such as the GTX 10xx or P40, change the llama-swap image in the Compose file to ghcr.io/mostlygeek/llama-swap:unified-cuda.

After startup, confirm that llama-swap is healthy before opening the dashboard:

docker compose -f docker-compose.amd.yaml ps

Use the Compose filename you started. If a service does not start, inspect its logs with docker compose -f docker-compose.amd.yaml logs llama-swap or docker compose -f docker-compose.amd.yaml logs llama-dash.

The default image tag is latest. To pin a deployment, set LLAMA_DASH_IMAGE_TAG to a published image tag, such as a commit SHA tag. Contributors can build llama-dash from a local checkout by adding -f docker-compose.build.yaml and --build, for example:

docker compose -f docker-compose.amd.yaml -f docker-compose.build.yaml up -d --build

Open the dashboard

Visit http://localhost:3000. On first launch, create the dashboard user. Then open Models, load your configured model, and follow Connect a client to send a request. Database migrations run automatically when llama-dash starts.

llama-swap is not exposed on the host in the bundled compose setup — llama-dash is the single public entrypoint on port 3000.

To pull newer images later, see Updating.

Next steps

Connect a client

Send your first request through the gateway.

Configuration

Environment variables and where things live.

Last updated on

llama.cpp & llama-swap

What llama-dash needs from the inference backend, and how much of it you have to set up yourself.

Manual setup

Run a production build of llama-dash against an existing llama-swap instance.

On this page

Before you start1Get the Compose files2Configure the environment3Create the llama-swap config4Start the stack5Open the dashboardNext steps