Hermes Voice
TaylorFinklea/hermes-voice
Interface iOS por voz para agentes de codificação locais com backend FastAPI
Ver no GitHubAgent iOS nativo para voz que conecta um iPhone a um agente de codificação local (Hermes, Claude Code, etc.) em um Mac, via backend FastAPI sobre Tailscale. Funciona com push-to-talk, reconhecimento de fala, execução de código e síntese de voz em ciclos completos.
Instalação
npx convez-ai TaylorFinklea/hermes-voice1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Executar comandos de codificação no Mac via voz do iPhone
- Manter estado de ferramentas e contexto em múltiplas rodadas de interação
- Substituir interfaces de chat de voz precárias com aplicativo dedicado
Veredito

Projeto técnico robusto com arquitetura bem definida (STT → LLM → TTS), suporte a múltiplos provedores e provisionamento seguro via Tailscale. Ideal para quem quer interface dedicada por voz para agentes locais, mas exige infraestrutura macOS e conhecimento DevOps.
README
A tradução para o português deste README ainda está na fila.
Harness Voice
Voice-native iPhone front-end for a local coding agent (Hermes, Claude Code, Codex, or OpenCode). Push-to-talk on the phone, Hermes does the actual operating on the Mac, audio answer plays back. Built to replace the rough Telegram-voice experience with something dedicated.
Architecture
┌──────────────┐ audio (m4a) ┌──────────────────┐ subprocess ┌───────────┐
│ iOS app │ ──────────────▶ │ FastAPI backend │ ─────────────▶ │ hermes CLI│
│ (SwiftUI) │ │ /api/audio │ │ on macOS │
│ │ ◀────────────── │ /api/text │ ◀───────────── │ │
└──────────────┘ TurnResponse │ STT + TTS layer │ stdout/ └───────────┘
(text + URL) └──────────────────┘ session_id
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
STT providers TTS providers Audio cache
(OpenAI, Groq, (ElevenLabs, (tmp dir,
faster-whisper) OpenAI, Piper) served by id)
A turn is one round trip: iPhone records → POSTs audio → backend STT → hermes chat -Q -q <text> (with --resume <id> on follow-ups so Hermes keeps tool/context state) → backend TTS → iPhone plays mp3/wav.
What's in the box
backend/— FastAPI service. Provider abstractions for STT (OpenAI / Groq / local faster-whisper / mock) and TTS (ElevenLabs / OpenAI / local Piper / mock). 18 unit tests with fake providers.ios/HermesVoice/— SwiftUI app, regenerable via XcodeGen fromproject.yml. Push-to-talk that supports both hold-to-talk and tap-to-latch. Always-visible text input fallback.
Quick start
1. Set up a Mac server (recommended)
From the app's Set up a Mac server screen, copy the versioned prompt into a local coding harness running on the target Mac. The prompt runs the first-party provisioner and pauses before it configures a persistent tailnet-only HTTPS proxy (Tailscale Serve) and installs one backend LaunchAgent. It never asks you to paste a token into the chat, and it never generates or requests a certificate file.
The provisioner supports these commands. Run them from the repository root—the
checkout directory containing backend/:
uv run --project backend --extra provision python backend/scripts/provision_macos.py setup
uv run --project backend --extra provision python backend/scripts/provision_macos.py setup --from-bws
uv run --project backend --extra provision python backend/scripts/provision_macos.py pair
setup discovers the checkout, uv, Tailscale, and MagicDNS name; binds the
backend to loopback; reuses or adds the exact Tailscale Serve HTTPS mapping to
127.0.0.1:8765; stores the server token in the macOS Keychain; installs one
io.harnessvoice.backend LaunchAgent; and checks the public HTTPS /health
endpoint before reporting ready. It never calls tailscale serve reset and
never touches an unrelated Serve mapping — a conflicting existing mapping on
port 8765 stops setup before any local change. --from-bws is an opt-in path
for a registered Bitwarden Secrets Manager project; the managed token is
passed to a child process through its environment only. If any step after the
mapping/LaunchAgent cutover fails (public health, authenticated harness
check), setup automatically rolls back everything it changed and restores
what was previously running; rerunning setup after fixing the underlying
problem is always safe.
pair shows a five-minute, one-time QR code, but only once every readiness
gate passes: a server token already stored in Keychain, an exact
non-conflicting Tailscale Serve mapping (or, for the advanced manual
direct-TLS runtime, both HERMES_VOICE_CERT and HERMES_VOICE_KEY set), a
public HTTPS /health that reports mock: false, and an authenticated
/api/harnesses listing showing Hermes available. Scan the code from the
app; the long-lived server token is never encoded in the QR code, and the
pairing secret is never printed separately from the code itself.
Troubleshooting guided setup
Guided setup supports macOS with Tailscale only. Run it from a local coding
harness on the target Mac and approve the Serve/LaunchAgent configuration
when prompted. If setup stops after a prerequisite changes, rerun setup; to
replace an expired or used pairing code, rerun pair. If camera access is
unavailable, keep using the app's manual URL/token or Bonjour paths.
Non-secret, non-destructive diagnostics:
tailscale serve status --json
launchctl print "gui/$(id -u)/io.harnessvoice.backend"
tail -F "$HOME/Library/Logs/Harness Voice/backend.log"
curl -s "https://$(tailscale status --json | jq -r '.Self.DNSName | rtrimstr(\".\")'):8765/health" | jq
tailscale serve status --json only reads Serve state — it never mutates
anything and is always safe to run before or after setup/pair. The health
endpoint should report status: ok, scheme: https, and mock: false.
Never print the Keychain token or paste credentials into a harness prompt,
diagnostic command, issue, or chat. The copied app prompt contains only the
public repository and provisioner command.
Advanced local development
For local-only development without a public Tailscale server:
cd backend
uv sync --extra dev
cp .env.example .env # optional; do not use this for guided setup
uv run python -m app # serves on http://127.0.0.1:8765
With no env vars set, the backend auto-engages mock mode. Smoke-test it with:
curl -s http://127.0.0.1:8765/health | jq
curl -s -X POST http://127.0.0.1:8765/api/text \
-H 'content-type: application/json' \
-d '{"text":"hello hermes"}' | jq
2. iOS app
cd ios/HermesVoice
xcodegen generate # regenerates HermesVoice.xcodeproj from project.yml
open HermesVoice.xcodeproj
Then in Xcode:
- Select a development team in Signing & Capabilities (your personal team is fine).
- Pick a simulator (or your iPhone) and Run.
- First launch on a real phone: tap the gear icon and set the backend URL to your Mac's reachable address (Tailscale IP recommended, e.g.
http://100.64.x.y:8765). - Tap Ping /health in Settings to verify connectivity.
3. Reach your Mac from your phone
You have three options, in order of preference:
- Tailscale (best). Install Tailscale on both, run the backend with
HERMES_VOICE_HOST=0.0.0.0, use your Mac's Tailscale IP in the iOS Settings screen. - LAN: same Wi-Fi, find your Mac's local IP (
ipconfig getifaddr en0), bind to0.0.0.0. - Simulator: leave host as
127.0.0.1— no extra config.
⚠️ When binding to
0.0.0.0, setHERMES_VOICE_TOKEN=<random>and put the same token in the iOS Settings auth field. The token is required on every request when set.
Env vars (see backend/.env.example)
| Variable | Default | What it does |
|---|---|---|
HERMES_VOICE_HOST | 127.0.0.1 | Bind address (use 0.0.0.0 over Tailscale/LAN only). |
HERMES_VOICE_PORT | 8765 | Listen port. |
HERMES_VOICE_MOCK | unset | Force mock mode even with keys present. |
HERMES_VOICE_TOKEN | unset | If set, requests must send X-Hermes-Voice-Token. |
HERMES_BIN | hermes | Path/name of the Hermes CLI. |
HERMES_EXTRA_ARGS | unset | Appended to every hermes chat (e.g. -t apple_notes,apple_reminders -m anthropic/claude-sonnet-4). |
HERMES_TIMEOUT_SECONDS | 180 | Per-turn timeout. |
STT_PROVIDER | auto | openai / groq / elevenlabs / local_whisper / mock. |
OPENAI_API_KEY / VOICE_TOOLS_OPENAI_KEY | unset | Used for OpenAI Whisper + OpenAI TTS. |
GROQ_API_KEY | unset | Groq Whisper-large-v3-turbo (very fast). |
LOCAL_WHISPER_MODEL | base.en | faster-whisper model name. |
TTS_PROVIDER | auto | elevenlabs / openai / local_piper / mock. |
ELEVENLABS_API_KEY | unset | ElevenLabs key (powers TTS and Scribe STT). |
ELEVENLABS_VOICE_ID | nPczCjzI2devNBz1zQrb | Brian (deeper narrator). |
OPENAI_TTS_MODEL | tts-1 | tts-1, tts-1-hd, or gpt-4o-mini-tts. |
OPENAI_TTS_VOICE | onyx | Masculine fallback voice. |
PIPER_VOICE_PATH | unset | Absolute path to a .onnx voice (with .onnx.json next to it). |
Auto-starting the backend
The guided setup command generates one portable, user-specific
io.harnessvoice.backend LaunchAgent under the current user's Library
directories and starts it only after confirmation. It binds the backend to
loopback and reuses Tailscale Serve for the tailnet-facing HTTPS proxy; it
never generates a certificate-renewal agent, because Tailscale Serve renews
its own certificate transparently. From the repository root—the checkout
directory containing backend/—run:
uv run --project backend --extra provision python backend/scripts/provision_macos.py setup
The checked-in files under backend/launchd/ are deprecation stubs and must not
be copied into ~/Library/LaunchAgents/. For advanced manual development,
continue using the loopback command in the section above; do not adapt the
checked-in stubs into a public service.
The iOS app's Settings → Diagnostics screen shows "Backend last seen" so you can spot the moment it goes unreachable.
Latency tuning
The slowest part of any turn is Hermes' LLM thinking + tool-call round-trips (2–6s typically). Beyond that, two knobs are worth knowing:
- STT provider: Groq's Whisper-large-v3-turbo is noticeably faster than
OpenAI's whisper-1 — often 200–400ms vs 600–1000ms for short utterances.
Set
STT_PROVIDER=groqandGROQ_API_KEY=...in.envto A/B. - TTS first-byte: ElevenLabs
eleven_turbo_v2_5delivers first audio in ~300ms. The_run_turnflow kicks off TTS streaming before the tool-call audit completes, so audit time doesn't push perceived audio start.
A/B-ing STT providers
If you have multiple keys set, the backend auto-picks openai first. To force a comparison without removing keys:
# in backend/.env
STT_PROVIDER=elevenlabs # Scribe v1 — strong WER, good with accents
# STT_PROVIDER=groq # whisper-large-v3-turbo — fastest by wall clock
# STT_PROVIDER=openai # whisper-1 — the safe default
# STT_PROVIDER=local_whisper # offline, requires '.[local]'
Restart the backend and call /health to confirm: the stt.name field reports which provider is live.
Scribe v2 streaming (the WebSocket "Scribe Realtime" product) is the upgrade path when we add streaming STT — it slots into the same STTProvider Protocol.
Fully local mode (no API calls)
cd backend
uv sync --extra dev --extra local
# Download a Piper voice (one-time, ~60MB):
mkdir -p ~/.local/share/piper-voices && cd ~/.local/share/piper-voices
curl -L -o en_US-ryan-high.onnx \
https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/ryan/high/en_US-ryan-high.onnx
curl -L -o en_US-ryan-high.onnx.json \
https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/ryan/high/en_US-ryan-high.onnx.json
Then in .env:
STT_PROVIDER=local_whisper
TTS_PROVIDER=local_piper
PIPER_VOICE_PATH=/Users/you/.local/share/piper-voices/en_US-ryan-high.onnx
Notes:
faster-whisperruns on CPU with int8 quantization on Apple Silicon — fine for short utterances. (Metal is not yet supported by CTranslate2.)Parakeetwould be the upgrade path for STT but its Apple Silicon story is still rough. There's a# TODOmarker inbackend/app/stt/local_whisper.pywhere the swap-in would go.- First Whisper transcription downloads the model (~150MB for
base.en).
Tests
cd backend
uv run pytest -q
Provider abstractions are unit-testable without network: tests inject FakeHermes and FakeTTS directly into the app factory.
Current limitations
- No streaming. v1 records the full utterance, sends after stop, gets one complete response, plays it. Real-time partial transcripts and streaming TTS are roadmap items.
- No interruption / barge-in. While Hermes is speaking, the mic button is disabled.
- No wake word / VAD. Push-to-talk only.
- Session continuity is best-effort. The backend stores
session_idround-trips, but if Hermes wipes a session, the next turn starts fresh. - No approval cards. Hermes runs whatever tools you have configured. Use
HERMES_EXTRA_ARGSto restrict toolsets (e.g.-t apple_notes,apple_remindersfor read-mostly).
Roadmap
- WebSocket transport for the iOS ↔ backend leg (replace HTTP POSTs).
- Streaming STT — partial transcripts visible in the transcript as you speak.
- ElevenLabs streaming TTS — playback starts before synthesis finishes.
- Interruption / barge-in — tapping the mic while Hermes speaks cuts him off and starts recording.
- Approval cards — backend surfaces "Hermes wants to send iMessage to Dad: ...", iOS shows an Approve/Deny card before the action runs.
- iOS Shortcut: "Hey Siri, ask Hermes ..." → fires
/api/textand speaks the reply. - Home Assistant voice-satellite integration — share the same backend with HA Assist.
- Background audio session improvements (currently playback uses
.duckOthers). - Swap faster-whisper for NVIDIA Parakeet (
parakeet-mlx) once it's plug-and-play on Apple Silicon.
Layout
hermes-voice/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI app + route registration
│ │ ├── config.py # env-driven Settings
│ │ ├── models.py # request/response shapes
│ │ ├── hermes.py # subprocess-based HermesClient + MockHermesClient
│ │ ├── audio_store.py # short-lived synthesized-audio cache
│ │ ├── stt/ # provider Protocol + openai/groq/local/mock impls
│ │ └── tts/ # provider Protocol + elevenlabs/openai/piper/mock impls
│ ├── tests/ # pytest (no network, all-fake providers)
│ ├── pyproject.toml
│ └── .env.example
├── ios/HermesVoice/
│ ├── project.yml # XcodeGen project spec
│ └── HermesVoice/
│ ├── HermesVoiceApp.swift
│ ├── Models/ # Message, AppSettings
│ ├── Services/ # VoiceRecorder, AudioPlayer, HermesVoiceAPI
│ ├── ViewModels/ # ConversationViewModel (state machine)
│ └── Views/ # MainView, TranscriptView, PushToTalkButton, SettingsView
└── README.md