Therfour

Multilingual Voice Agent for Harm Reduction

Therfour is a modular, open-source backend for telephone-based harm-reduction and harm-prevention helplines. It connects to a phone call via Twilio Media Streams, runs all AI inference locally, and returns synthesised speech – no data ever leaves your infrastructure.

flowchart TD
     Caller[Caller] --> Twilio[Twilio]
     Twilio --> Inbound["/calls/inbound (TwiML)"]
     Inbound --> Stream["WebSocket /calls/stream"]

     subgraph Session[CallSession]
          direction TB
          Decode["mu-law 8 kHz -> float 16k"]
          STT["faster-whisper"]
          LLM["Ollama LLM"]
          TTS["Piper TTS"]
          Encode["float 22k -> mu-law 8 kHz"]
          Decode --> STT --> LLM --> TTS --> Encode
     end

     Stream --> Decode
     STTHint[STT] -.-> STT
     LLMHint["Local LLM (harm-reduction prompt)"] -.-> LLM
     TTSHint[TTS] -.-> TTS
     Encode --> ReturnAudio[Audio sent back to caller]

Loading

Tech stack

Layer	Library / Tool
Web server	FastAPI + Uvicorn
Telephony	Twilio Media Streams
STT	faster-whisper
TTS	Piper
LLM	Ollama (local, any model)
Audio codec	Python audioop / audioop-lts + SciPy

Quick start

Prerequisites

• Python 3.11+
• Piper binary on your $PATH
• Ollama running locally with your chosen model pulled
• A Twilio account with a voice-capable phone number

1 – Install Python dependencies

pip install -r requirements.txt

2 – Download Piper voice models

mkdir -p models
# English (US) – libritts_r medium voice (default)
wget -q https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/libritts_r/medium/en_US-libritts_r-medium.onnx \
     -O models/en_US-libritts_r-medium.onnx
wget -q https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/libritts_r/medium/en_US-libritts_r-medium.onnx.json \
     -O models/en_US-libritts_r-medium.onnx.json

# English (US) – amy medium voice
wget -q https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/amy/medium/en_US-amy-medium.onnx \
     -O models/en_US-amy-medium.onnx
wget -q https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/amy/medium/en_US-amy-medium.onnx.json \
     -O models/en_US-amy-medium.onnx.json

3 – Pull an Ollama model

ollama pull llama3.2:3b   # ~2 GB; swap for any model you prefer

4 – Configure environment

cp .env.example .env
# Edit .env – at minimum set TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, PUBLIC_HOST

5 – Start the server

uvicorn app.main:app --reload

Expose the server to the internet (e.g. via ngrok) and configure your Twilio phone number to send voice webhooks to https://<your-host>/calls/inbound.

Docker Compose

cp .env.example .env   # edit as above
docker compose up --build

Ollama and the application container are started together. Pull your model inside the ollama container after first boot:

docker compose exec ollama ollama pull llama3.2:3b

Project structure

app/
├── main.py                  # FastAPI application
├── core/
│   └── config.py            # Pydantic-settings configuration
├── models/
│   └── schemas.py           # Shared Pydantic schemas
├── api/routes/
│   ├── health.py            # GET /health
│   └── calls.py             # POST /calls/inbound  WS /calls/stream
└── services/
    ├── stt.py               # Speech-to-text  (faster-whisper)
    ├── tts.py               # Text-to-speech  (Piper)
    ├── llm.py               # LLM generation  (Ollama)
    └── telephony.py         # Audio pipeline + CallSession orchestrator
tests/
├── test_api.py
├── test_stt.py
├── test_tts.py
├── test_llm.py
└── test_telephony.py

Running tests

pytest

Swift migration (in progress)

To start moving server-side logic from Python to Swift, this repository now includes a small Swift package at swift-backend/ that mirrors shared response models and the /health payload contract.

Run Swift tests:

cd swift-backend
swift test

Configuration reference

All settings can be overridden via environment variables or a .env file. See .env.example for the full list with descriptions.

Variable	Default	Description
WHISPER_MODEL	small	faster-whisper model size
WHISPER_LANGUAGE	(auto-detect)	Pin transcription language
PIPER_BINARY	piper	Path to the Piper executable
PIPER_MODEL_PATH	models/en_US-libritts_r-medium.onnx	Piper fallback voice model path
PIPER_DEFAULT_VOICE_ID	en-US-libritts-r-medium	Default Piper voice id
PIPER_VOICES_CONFIG_PATH	app/core/piper_voices.json	Piper voice catalog config
OLLAMA_MODEL	llama3.2:3b	Ollama model tag
OLLAMA_BASE_URL	http://localhost:11434	Ollama API base URL
SILENCE_TIMEOUT_S	1.5	Seconds of silence before turn processing
PUBLIC_HOST	localhost	Hostname used in the TwiML <Stream> URL