Open-LLM-VTuber

Overview

Local AI companion system running on the desktop (10.100.0.2). Frontend is an Electron + React app (github.com/yuuhikaze/Open-LLM-VTuber-Web). Backend is a Python FastAPI server (github.com/yuuhikaze/Open-LLM-VTuber).

Component map:

NixOS Integration

nix-ld

modules/features/nix-ld.nix enables nix-ld with CUDA and audio libraries so Electron’s prebuilt binary resolves native dependencies at runtime.

Hyprland Window Rules

modules/features/hyprland.nix contains windowrulev2 rules for class: open-llm-vtuber:

These make the Live2D overlay render correctly as a pinned transparent window in pet mode.

Ollama

modules/features/ollama.nix configures ollama-cuda listening on 0.0.0.0:11434 with gemma4:e4b as the active model.

Firewall

modules/features/open-llm-vtuber.nix adds an nftables rule allowing TCP port 12393 only from the laptop’s WireGuard IP (10.100.0.3). All other sources are blocked. No other hosts can reach the backend.

Running the Backend

The full stack (Letta, Qwen3-TTS, backend) is managed by the companion command — companion for kitty tabs, companion start|stop|status headless, remote control through Discord (reference yobidashi). Manual run for development:

uv sync
uv pip install faster-whisper
LD_LIBRARY_PATH=/run/opengl-driver/lib uv run run_server.py

The LD_LIBRARY_PATH prefix exposes the NVIDIA CUDA driver libraries (libcuda.so, libcudart.so) to Python’s dynamic linker, enabling GPU-accelerated ASR (faster-whisper) and TTS. NixOS places these libraries under /run/opengl-driver/lib which is not in the default linker search path. The env var is scoped to a single command — it does not persist in the shell.

The server binds to 0.0.0.0:12393.

Key Configuration (conf.yaml)

Character:

Conversation agent:

ASR:

TTS:

Running the Frontend

The frontend uses a Nix flake with direnv for automatic environment setup:

bun run dev

direnv loads the flake automatically on cd, providing a NixOS-patched Electron binary via ELECTRON_EXEC_PATH.

Pet mode: click the layers icon in the sidebar and select “Pet Mode”. The window becomes a transparent overlay pinned above all other windows. Hyprland window rules apply automatically based on the window class.

Configuration

Settings are saved to localStorage and persist across sessions per workstation.

Desktop

Laptop

Live2D Models

Available models are declared in model_dict.json at the repo root:

Model files live under live2d-models/ in the backend repo.

Tingyun expression indices (mapped in model_dict.json, correspond to the Expressions array in the model’s model3.json):

To add a model:

  1. Copy the model folder to live2d-models/
  2. Add an entry to model_dict.json
  3. Set live2d_model_name in conf.yaml