Open WebUI

Overview

Open WebUI running on the server, accessible at chat.yhkze.net. Chat interface for LLMs with text and image input, replacing ad-hoc use of chatgpt.com.

Module: modules/features/open-webui.nix

Most settings categories (connections, web search, and more) seed into Open WebUI’s own SQLite db on first boot and ignore env var changes after that — Nix only configures what’s always env-driven regardless (OAuth). Everything else is managed through Admin Panel → Settings, documented below so it’s reproducible if the db is ever lost.

Authentication

Keycloak OIDC (realm internal), not the built-in signup flow (ENABLE_SIGNUP=false). Open WebUI’s role management reads the groups claim on every login:

Keycloak Group Open WebUI Role
open_webui_admin Admin
open_webui User
any other group Refused (403)
no group Pending, until an admin approves

Because the role is re-evaluated on every login, an admin who leaves open_webui_admin is demoted the next time they sign in.

Admin UI Configuration

Everything below lives in Open WebUI’s db, not in this repo — configured once through the web UI as the admin account.

Connections (Admin Panel → Settings → Connections), both OpenAI-compatible:

Name Base URL Notes
OpenRouter https://openrouter.ai/api/v1 Has usage limits/cost — not the default model, to avoid burning credits by accident.
USFQ AI https://ai.usfq.edu.ec/api/agents/llm/main/v1 Work-provided, effectively unlimited; serves zai-org/GLM-5.3-Flash.

Desktop’s Ollama instance is not wired in yet — its SSD failed, so there is no local-model backend until it’s rebuilt.

Default model: zai-org/GLM-5.3-Flash (USFQ), set via the model picker’s “Set as default” — every new chat starts on this model rather than an OpenRouter one.

Web Search (Admin Panel → Settings → Web Search): toggled on, engine set to searxng, query URL http://127.0.0.1:8096/search?q=<query> — see SearXNG. The env vars that would normally seed this (ENABLE_RAG_WEB_SEARCH, RAG_WEB_SEARCH_ENGINE, SEARXNG_QUERY_URL) don’t work here: web search lives under the same rag.* settings bucket Open WebUI’s own content-extraction defaults already seeded on first boot, so the category was never “untouched” the way a fresh install’s would be.

Context compaction (Admin Panel → Settings → Interface): on, so older messages are summarized by the chat’s own model once a chat’s context passes a token threshold. The threshold applied is the selected model’s compact_token_threshold param (Admin Panel → Settings → Models → Advanced Params) or, when unset, the global one, clamped to the cap. Retention counts messages, not tokens, and rounds to a user-message boundary.

Setting Value Reason
Token Threshold 96000 Fallback for models without their own value; fits any window of 128k or more
Token Cap 800000 About 80% of the 1M-window models
Retained Messages 25 Last quarter kept verbatim; the chat grows about 3x before the next compaction
Context Compaction Model Current Model The summary call fits the model that holds the chat

Models with a larger window than about 128k set compact_token_threshold to 75-80% of it, leaving room for the reply and tool output within one turn. zai-org/GLM-5.3-Flash reports max_model_len 262144 in its /models entry and has 204800. A workspace model doesn’t inherit its base model’s params: the Kioku model takes the same value from kioku’s publish step.

Code Execution

The model can run code through Open Terminal, a REST shell served from a sandbox container on the server. It supports any language, so answers can be checked by running them instead of guessed. The sandbox ships only nix, python3, git and curl; anything else is fetched on demand:

nix run nixpkgs#hello
nix shell nixpkgs#gcc nixpkgs#rustc -c bash

Module: modules/features/open-terminal.nix

Registration (Admin Panel → Settings → Integrations → Terminals), stored in Open WebUI’s db as terminal_server.connections:

Field Value
URL http://127.0.0.1:8098
Auth Bearer, open-terminal/api-key from SOPS

Register it as an admin-level connection, not under a user’s own Settings → Integrations. Admin connections are proxied by the Open WebUI backend, which is why a loopback URL works; a per-user connection is called from the browser, where 127.0.0.1 is the client machine, and chats fail with Terminal server 'http://127.0.0.1:8098' not found because the backend has no connection with that id.

The Kioku model, created by kioku’s publish script, gets the terminal on every request (reference kioku MCP); for any other model, enable it per chat from the chat’s tool menu.

Isolation — the container runs on its own bridge (ot0, 10.89.50.0/24) with all capabilities dropped and CPU, memory and PID limits. An nftables table lets it reach the internet but drops traffic to RFC1918, link-local and CGNAT destinations (LAN, WireGuard, Mullvad link, other container networks) and to every host service except podman’s DNS resolver. The host’s name overrides (such as git.yhkze.net pointing at 10.100.0.1) do not apply: the sandbox has no /etc/hosts copy and queries a public resolver, so it reaches the server’s names over the internet. Egress also requires kernel forwarding on ot0, enabled in the module because sysctl.nix disables forwarding on every interface not listed and each rebuild re-applies that.

Persistence — the /nix podman volume keeps fetched packages across restarts. /var/lib/open-terminal/work (mounted at /work) holds files shared with the model and is included in the restic backup. The volume shadows the image’s /nix, so before each start the unit copies the closure behind the image’s /bin and /usr/bin links into it; without that, a newer image or a volume restored on reinstall leaves /bin/sh dangling and uv exits with Failed to find any common binaries to determine libc. Nix builds run with sandbox = false since the container has no capabilities, so substitution from the binary cache works but source builds are unsandboxed.

kioku Tools

Anki and Trilium tools come from the kioku MCP server, registered as an admin tool server connection at http://127.0.0.1:8099/mcp, and the kioku conventions and diagram rules are published as skills, defaults of the Kioku model. Images the model draws in the terminal’s /work reach Anki and Trilium without leaving the server. Reference kioku MCP for this deployment and the kioku repo’s docs for the Open WebUI setup.

Operations

systemctl status open-webui
journalctl -u open-webui -n 50

The package comes from the open-webui flake input, which tracks nixos-unstable and moves only with nix flake update open-webui. Releases migrate the database schema on first start, so take a cold copy before deploying one; a failed migration stops startup at the migration error.

sudo systemctl stop open-webui
sudo cp -a /var/lib/open-webui /persist/backup/open-webui-$(date +%F)

rebuild 10.100.0.1 then starts the new release. To roll back, revert flake.lock, rebuild, and restore the copy with the service stopped.