Rules assistant (RAG)

The rules assistant lets you ask natural-language questions about a game’s uploaded rulebook(s) — for example “how many cards do you start with?” — and get an answer with page citations. Answers are based on passages from the manuals you uploaded and cite the pages they come from, so you can check them.

By default it runs locally through Ollama, so your rulebooks and questions stay on your server. No API keys and no cloud services are required.

Each user can ask 10 questions per minute. Readers cannot use the assistant.

Enabling it

If you downloaded the docker-compose.yml from the repository, it already has an ollama service behind the rag profile. Put RAG_ENABLED=true in the .env file next to the compose file and start the stack with that profile:

docker compose --profile rag up -d

A plain docker compose up -d does not start Ollama, so keep RAG_ENABLED off in that case.

The compose file on the Docker page doesn’t include the AI service. To turn the assistant on there, add an ollama service to your docker-compose.yml and point the app at it:

services:
  boardgametracker:
    environment:
      - RAG_ENABLED=true
      - AI_BASE_URL=http://ollama:11434
      - AI_CHAT_MODEL=qwen3:4b

  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    volumes:
      - ./ollama:/root/.ollama

Keep your existing environment entries and add these three. Then (re)start the stack:

docker compose up -d

The required models (bge-m3 for search, plus your AI_CHAT_MODEL) are pulled when the application starts, and re-checked on every restart, so no manual ollama pull is needed. Changing the chat model later only needs a redeploy.

Once enabled, every uploaded manual is indexed in the background. Manuals uploaded before enabling are indexed on the next start.

Models

Role	Model	Where it runs
Embeddings (search)	`bge-m3` (fixed)	Always on local Ollama. Chosen for multilingual rulebooks (English, Dutch, French, German, …).
Chat (answers)	`qwen3:4b` by default	Local Ollama or a hosted provider; set `AI_CHAT_MODEL` / `AI_PROVIDER`.

The embedding model is fixed at bge-m3 (1024 dimensions) so the vector search stays consistent with what’s already stored in the database; it always runs locally. The chat model is independent: run it locally, or point it at a hosted provider (see below) while embeddings stay on your server.

Choosing a chat model

Larger models give better answers but need more memory, and ideally a GPU. Set your choice with AI_CHAT_MODEL.

Your hardware	Suggested `AI_CHAT_MODEL`
Raspberry Pi / under 8 GB RAM	Not recommended — keep the assistant disabled.
~8 GB RAM, CPU only	`qwen3:4b` (default) or `mistral:7b`
16 GB RAM, or a 6–8 GB GPU	`qwen3:8b`
12–16 GB GPU	`qwen3:14b`, `gemma3:12b`, or `qwen3:30b-a3b`
24 GB+ GPU	`command-r`

The bge-m3 embedding model runs fine on CPU across the board.

GPU acceleration (NVIDIA)

By default Ollama runs on the CPU. To use an NVIDIA GPU for faster answers:

Install the NVIDIA Container Toolkit on the host.
Download docker-compose.gpu.yml and the .env file from the quick start. It is the same stack with Ollama always started and the GPU reserved for it. It also needs Docker Compose 2.24 or newer. RAG_ENABLED defaults to true in this file, but a .env copied from .env.example sets it to false, so change that line to RAG_ENABLED=true.

Start it:

docker compose -f docker-compose.gpu.yml up -d

The part that reserves the GPU is the deploy block on the ollama service:

  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    volumes:
      - ./ollama:/root/.ollama
    healthcheck:
      test: ["CMD", "ollama", "list"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

The GPU file also uses qwen3:4b unless you choose another model. For a 6–8 GB card set AI_CHAT_MODEL=qwen3:8b in .env.

A 6 GB card such as a GTX 1060 can run qwen3:8b, but not while the embedder is also on the GPU: the chat model needs roughly 5.5 GB and bge-m3 another ~0.6 GB, so the two don’t both fit in 6 GB. Pin the embedder to the CPU (see below) and the chat model gets the whole card.

Small GPUs: keep the embedder off the card

On a tight GPU (around 4 GB) a local chat model and the bge-m3 embedder compete for memory. When they don’t both fit, Ollama offloads part of the chat model to the CPU and generation slows down sharply. Because the embedder is small and only handles one short query per question, the fix is to pin it to the CPU so the chat model keeps the whole GPU:

AI_EMBEDDING_NUM_GPU=0

With this set, the chat model stays resident on the GPU across questions; only the first answer after startup pays the load time. The default (-1) lets Ollama decide, which is right when the GPU has room to spare (a bigger card, or a hosted chat model that leaves the GPU free).

Rulebook page images

Each answer’s citations include a thumbnail of the rulebook page they were drawn from. Pages are rendered on the server on demand and cached. No configuration is needed: the container ships with the renderer (poppler-utils). If the renderer is unavailable, answers fall back to text-only citations.

Using a hosted chat model

You can keep embeddings local while sending the chat step to a hosted, OpenAI-compatible provider, for stronger answers than your hardware can run locally. Only the chat model is hosted; bge-m3 embeddings and the vector search always stay on your server. Use the exact model id from your provider.

OpenAI:

AI_PROVIDER=openai
AI_BASE_URL=https://api.openai.com/v1
AI_CHAT_MODEL=gpt-4o-mini
AI_API_KEY=<your-openai-api-key>

Claude (Anthropic), through its OpenAI-compatible endpoint:

AI_PROVIDER=openai
AI_BASE_URL=https://api.anthropic.com/v1
AI_CHAT_MODEL=claude-sonnet-5-5
AI_API_KEY=<your-anthropic-api-key>

You still need the local Ollama service for embeddings (it only has to run bge-m3, which is small). With a hosted chat model the GPU is used for embeddings alone, or you can skip a GPU entirely.