Jump to section

Or jump from this list on a smaller screen.

This is a page from the Audivra documentation. Developer exports — index: http://127.0.0.1:3000/docs/llms.txt, full corpus: http://127.0.0.1:3000/docs/llms-full.txt, OpenAPI: http://127.0.0.1:3000/docs/openapi.json, Postman: http://127.0.0.1:3000/docs/postman.json. Press ⌘K to search docs.

Self-host on an RTX 4090

Run Ollama, Kokoro-82M, and MuseTalk on a local RTX 4090 — route short prompts on-box before cloud GPU offload.

Self-hosters

~30 min

Outcome: A local inference stack on :8090 / :11435 that handles TTS and lip-sync under ~500 characters without cloud credits.

1
Hardware and VRAM budget
RTX 4090 (24 GB VRAM) comfortably runs MuseTalk lip-sync (~4 GB), Kokoro TTS (~500 MB), and Ollama qwen2.5:3b (~2.5 GB) concurrently.
RTX 3060 (12 GB) works for TTS-only or sequential jobs — avoid parallel MuseTalk + large LLM on 8 GB cards.

Install dependencies

Clone the repo, copy .env.example to .env, and install Python inference deps plus Ollama.

Terminal

git clone https://github.com/your-org/audivra.git && cd audivra
cp .env.example .env

# LLM for fast-path (<500 chars)
ollama pull qwen2.5:3b

# Python inference (Kokoro + MuseTalk)
pip install -r inference/requirements.txt
npm run inference:bootstrap

3
Start local services
Run inference on :8090 (TTS + lip-sync) and the local worker gateway on :11435 (routes Ollama + Kokoro before cloud).
Terminal — two processes
```
# Terminal 1 — GPU pipeline
npm run inference          # :8090

# Terminal 2 — fast-path gateway
npm run local-worker       # :11435
```

Wire Audivra to your local stack

Point the Next.js app or gateway at your local worker. Short prompts route locally; longer scripts offload to Modal/cloud.

.env

OLLAMA_URL=http://127.0.0.1:11434
LOCAL_WORKER_GATEWAY_URL=http://127.0.0.1:11435
INFERENCE_URL=http://127.0.0.1:8090
FAST_PATH_CHAR_LIMIT=500
LOCAL_VRAM_THRESHOLD_GB=6

5
Smoke-test the fast path
Send a short TTS request through the unified generate endpoint — meta.executedBy should return local_worker with 1 routing credit.
cURL
```
curl -X POST http://localhost:3000/api/v1/generate \
  -H "Authorization: Bearer sk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello from my RTX 4090.","taskType":"tts","voice_id":"aurora"}'
```
Lip-sync video jobs over ~200 characters still route to gpu_cluster even on a healthy local worker.

Self-host on an RTX 4090

Hardware and VRAM budget

Install dependencies

Start local services

Wire Audivra to your local stack

Smoke-test the fast path

Related reference

Local worker reference

Deploy to production

Processing nodes