We fit a full offline voice agent into 1.2 GB on Android
88-second demo — watch on YouTube
The loop
Four small task-specific models run in sequence, all on CPU:
| Stage | Model | Download | Runtime |
|---|---|---|---|
| Voice activity | Silero VAD v5 | 2 MB | ONNX Runtime |
| Streaming STT + end-of-utterance | Parakeet-EOU 120M INT8 | 153 MB | ONNX Runtime |
| Tool calls | FunctionGemma 270M | 327 MB base + 9.5 MB LoRA adapter | LiteRT-LM |
| Streaming TTS | Pocket TTS 100M INT8 | ~126 MB | ONNX Runtime |
The router is not the stock model: we LoRA-tuned a 9.5 MB Control adapter on the exact tool
schema and a compact serialization of device state, and at runtime the app offers
FunctionGemma only the tools valid in the current state (you cannot stop_music if nothing
is playing). There is no keyword or regex fallback — routing is entirely the model's
decision. Pocket TTS emits fixed 80 ms frames, so playback starts while the reply is still
being synthesized.
Measured on the phone
Galaxy S23 Ultra, 2026-07-17, speech-android f01841a, CPU only:
| Metric | Value |
|---|---|
| STT (end of speech → final transcript) | 217 ms |
| FunctionGemma tool call (12-run mean, routing-only decoding) | 294 ms |
| TTS first audio | 179 ms |
| End of speech → first spoken audio | 908 ms |
| Resident app memory (PSS) | 1,116 MB |
Why it fits
Three decisions did most of the work:
- Small task-specific models instead of one big one. A 120M recognizer, a 270M router, and a 100M synthesizer together download about 620 MB. None of them needs a GPU.
- INT8 everywhere the quality survives. The STT and TTS graphs are INT8 ONNX exports; FunctionGemma runs through LiteRT-LM as a base bundle plus the 9.5 MB LoRA Control adapter.
- Sequential execution with bounded context. The stages run one after another, so their peaks do not stack, and the LLM sees a compact serialized state instead of a chat history.
The pipeline orchestration (turn detection, barge-in, speech queue) is speech-core, C++17. The Android layer is a thin Kotlin SDK over a ~250-line JNI bridge. Everything is Apache-2.0.
Try it
Signed APK:
https://github.com/soniqo/speech-android/releases/latest/download/control-demo-release.apk
(models download on first run, ~620 MB). Or build the control-demo module from source:
https://github.com/soniqo/speech-android
The current tool surface is deliberately small — volume, media, a few device toggles. What would you actually want a phone agent to control first?