Local AI companion system running on the desktop (10.100.0.2). Frontend is an Electron + React app (github.com/yuuhikaze/Open-LLM-VTuber-Web). Backend is a Python FastAPI server (github.com/yuuhikaze/Open-LLM-VTuber).
Component map:
gemma4:e4b; also provides embeddings
(nomic-embed-text) for Letta’s archival memoryopenrouter/… handles), freeing the GPU for ASR and
TTSmodules/features/nix-ld.nix enables nix-ld with CUDA and
audio libraries so Electron’s prebuilt binary resolves native
dependencies at runtime.
modules/features/hyprland.nix contains
windowrulev2 rules for
class: open-llm-vtuber:
noblur, noshadow,
opacity 1.0 override, float,
pinThese make the Live2D overlay render correctly as a pinned transparent window in pet mode.
modules/features/ollama.nix configures
ollama-cuda listening on 0.0.0.0:11434 with
gemma4:e4b as the active model.
modules/features/open-llm-vtuber.nix adds an nftables
rule allowing TCP port 12393 only from the laptop’s WireGuard IP
(10.100.0.3). All other sources are blocked. No other hosts
can reach the backend.
The full stack (Letta, Qwen3-TTS, backend) is managed by the
companion command — companion for kitty tabs,
companion start|stop|status headless, remote control
through Discord (reference yobidashi).
Manual run for development:
uv sync
uv pip install faster-whisper
LD_LIBRARY_PATH=/run/opengl-driver/lib uv run run_server.pyThe LD_LIBRARY_PATH prefix exposes the NVIDIA CUDA
driver libraries (libcuda.so, libcudart.so) to
Python’s dynamic linker, enabling GPU-accelerated ASR (faster-whisper)
and TTS. NixOS places these libraries under
/run/opengl-driver/lib which is not in the default linker
search path. The env var is scoped to a single command — it does not
persist in the shell.
The server binds to 0.0.0.0:12393.
conf.yaml)Character:
live2d_model_name: tingyunhuman_name: StevenConversation agent:
conversation_agent_choice: letta_agent — conversations
run through a self-hosted Letta server (long-term memory); setup
documented in Lettahost: 127.0.0.1 (not localhost — IPv6
breaks the SSE stream under uvloop)faster_first_response: false — with streamed TTS a
longer first sentence masks the next sentence’s synthesis prefill,
avoiding an audible gap after the openerollama/qwen2.5:3b or
openrouter/openai/gpt-4o-mini), not in
conf.yaml. When using OpenRouter handles, the
OPENROUTER_API_KEY env var must be set on the Letta
containerASR:
faster_whisperlarge-v3-turboLD_LIBRARY_PATH=/run/opengl-driver/lib; falls back to CPU
if the env var is not setTTS:
vllm_omni_tts backed by a vLLM-Omni container
serving Qwen3-TTSvoice_clone (reference audio + transcript);
the audio directory is mounted into the container at
/audio/[stream-audio] underrun #NThe frontend uses a Nix flake with direnv for automatic environment setup:
bun run devdirenv loads the flake automatically on cd, providing a
NixOS-patched Electron binary via ELECTRON_EXEC_PATH.
Pet mode: click the layers icon in the sidebar and select “Pet Mode”. The window becomes a transparent overlay pinned above all other windows. Hyprland window rules apply automatically based on the window class.
Settings are saved to localStorage and persist across sessions per workstation.
ws://127.0.0.1:12393/client-wshttp://127.0.0.1:123931.0 — JPEG quality of
camera/screen capture frames sent to the vision LLM. Default is
0.8; 1.0 avoids compression artifacts that can
confuse the model502080ws://10.100.0.2:12393/client-wshttp://10.100.0.2:123931.0502080Available models are declared in model_dict.json at the
repo root:
mao_protingyun (active)Model files live under live2d-models/ in the backend
repo.
Tingyun expression indices (mapped in
model_dict.json, correspond to the Expressions
array in the model’s model3.json):
To add a model:
live2d-models/model_dict.jsonlive2d_model_name in conf.yaml