convez.ai
PluginAutomaçãoIntermediário
EzekielOgunkunle

Voxpip

EzekielOgunkunle/voxpip

Transcreva vídeos localmente — sem API key, sem custos

Ver no GitHub
InicialPluginScore · de 0 a 100

Plugin Claude Code que transcreve vídeos localmente sem necessidade de API keys. Suporta transcrição via STT local (voxtype, whisper.cpp, mlx_whisper), com fallback para APIs em nuvem (Groq/OpenAI). Extrai frames inteligentemente e passa tudo ao Claude para análise contextualizada de conteúdo visual e áudio.

Instalação

$npx convez-ai EzekielOgunkunle/voxpip

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
2
Forks
1
Acompanhando
0
Saúde
42%
Último commit
há 6 meses
Versão
—
Issues abertas
0
Contribuidores
2

Casos de uso

  • Análise de conteúdo de vídeos do YouTube, Loom ou redes sociais
  • Extração de transcrições e insights de apresentações ou webinars
  • Processamento offline de vídeos sensíveis ou privados

Veredito

Solução prática que resolve um gap real: transcrição de vídeos sem custos de API. Arquitetura clara com fallbacks inteligentes (local STT → Groq → OpenAI). Recomendado para quem trabalha muito com vídeo e quer manter dados offline.

README

A tradução para o português deste README ainda está na fila.

/voxpip

Give Claude the ability to watch any video — even without an API key.

Claude Code:

/plugin marketplace add EzekielOgunkunle/voxpip
/plugin install voxpip@voxpip

claude.ai (web): install via the Claude Code plugin marketplace or clone the repo directly.

Codex / generic skills:

git clone https://github.com/EzekielOgunkunle/voxpip.git ~/.codex/skills/voxpip

Zero config to start — local STT detected automatically on first run.

Note: voxpip is a Claude Code plugin, not a PyPI package. pip install voxpip will not work — use the install commands above.


I built this because I work with a lot of video content and I wanted a way to hand video to Claude without needing an API key just to get a transcript. Brad Bonanno's /watch plugin was the perfect foundation — solid pipeline, clean code. I added a local STT tier so the transcription fallback works even if you've never touched a Groq or OpenAI account.

What it adds to /watch

This plugin forks bradautomates/claude-video. Brad's pipeline handles everything: download via yt-dlp, frame extraction via ffmpeg, native caption parsing, and cloud Whisper fallback (Groq/OpenAI). voxpip's single addition is a local STT tier that sits between caption parsing and the cloud APIs.

If a video has no native captions, voxpip tries your local STT engine before reaching for a cloud API. If local STT isn't installed either, it falls back to Groq or OpenAI exactly as the upstream does. You can also force the behavior with --no-local-stt or --no-whisper.

How it works

  1. You paste a video and a question. URL (anything yt-dlp supports — YouTube, Loom, TikTok, X, Instagram, plus hundreds more) or a local path (.mp4, .mov, .mkv, .webm).
  2. yt-dlp downloads it. For URLs, into a temp working directory. For local files, no download — just probed in place.
  3. ffmpeg extracts frames at an auto-scaled rate. Duration-aware budget: ≤30s gets ~30 frames, 30-60s gets ~40, 1-3min gets ~60, 3-10min gets ~80, longer gets 100 sparsely. Hard caps: 2 fps, 100 frames. JPEGs at 512px wide by default.
  4. The transcript comes from one of three places. First: native captions from yt-dlp (free, instant). Second: local STT engine on your machine (free, offline). Third: Whisper API — Groq's whisper-large-v3 (preferred) or OpenAI's whisper-1.
  5. Frames + transcript are handed to Claude. The script prints frame paths with t=MM:SS markers and the transcript with timestamps. Claude Reads each frame in parallel — JPEGs render directly as images in its context.
  6. Claude answers grounded in what's actually on screen and in the audio. Not "based on the description" or "according to the title." It saw the frames. It heard the transcript.
  7. Cleanup. The script prints a working directory at the end. If you're not asking follow-ups, Claude removes it.

Local STT engines

EnginePlatformInstall
voxtypeLinux (Arch)paru -S voxtype or yay -S voxtype — non-Arch: see voxtype.io
whisper.cppLinux/macOS/Windowsgithub.com/ggerganov/whisper.cpp — build or grab a release binary
mlx_whispermacOS (Apple Silicon)pip install mlx-whisper
whisper (openai-whisper)Linux/macOS/Windowspip install openai-whisper

voxtype note: voxtype is primarily a push-to-talk dictation tool. voxpip invokes it via voxtype transcribe <file> — voxtype must expose this CLI transcription mode to be usable here. See voxtype.io/docs to verify your version supports it.

setup.py detects whichever is on your PATH and caches the result in ~/.config/voxpip/.env (key: LOCAL_STT_ENGINE). Re-detection only happens if the key is missing.

Bring your own keys (optional)

CapabilityWhat you needCost
Local STT (auto-detected)voxtype / whisper.cpp / mlx_whisper / openai-whispernone required
Download + native captionsyt-dlp + ffmpegFree
Whisper fallback (preferred)Groq API key — whisper-large-v3Cheap, fast
Whisper fallback (alt)OpenAI API key — whisper-1Standard pricing
Disable Whisper entirely--no-whisperFree, frames-only when no captions

Install

SurfaceInstall
Claude Code/plugin marketplace add EzekielOgunkunle/voxpip then /plugin install voxpip@voxpip
claude.ai (web)Install via the Claude Code plugin marketplace
Codexgit clone https://github.com/EzekielOgunkunle/voxpip.git ~/.codex/skills/voxpip
Manual / devgit clone https://github.com/EzekielOgunkunle/voxpip.git ~/.claude/skills/voxpip

Usage

/voxpip https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?
/voxpip https://www.tiktok.com/@user/video/123 summarize this
/voxpip ~/Movies/screen-recording.mp4 when does the UI break?
/voxpip https://vimeo.com/123 what tools does she mention?

Focused on a specific section — denser frame budget, lower token cost:

/voxpip https://youtu.be/abc --start 2:15 --end 2:45
/voxpip video.mp4 --start 50 --end 60
/voxpip "$URL" --start 1:12:00            # from 1h12m to end

Other flags:

  • --no-local-stt — skip local STT tier, go straight to API fallback.
  • --max-frames N — lower the frame cap for a tighter token budget.
  • --resolution W — bump frame width to 1024 px when Claude needs to read on-screen text.
  • --fps F — override the auto-fps calculation (still capped at 2 fps).
  • --whisper groq|openai — force a specific Whisper backend.
  • --no-whisper — disable cloud transcription entirely.
  • --out-dir DIR — keep working files somewhere specific.

Structure

.
├── SKILL.md                 # skill contract — loaded by all three surfaces  [changed]
├── commands/
│   └── voxpip.md            # slash command shim                             [renamed from watch.md]
├── scripts/
│   ├── watch.py             # entry point — orchestrates download → frames → transcript
│   ├── download.py          # yt-dlp wrapper
│   ├── frames.py            # ffmpeg frame extraction + auto-fps logic
│   ├── transcribe.py        # VTT parsing + dedupe + Whisper orchestration
│   ├── whisper.py           # Groq / OpenAI clients (pure stdlib)
│   ├── setup.py             # preflight + installer  [changed: local STT detection]
│   └── build-skill.sh       # build dist/voxpip.skill for claude.ai upload
├── hooks/                   # SessionStart status hook (Claude Code only)
├── .claude-plugin/          # plugin.json + marketplace.json  [changed]
├── .codex-plugin/           # codex packaging
└── .github/workflows/       # release.yml — auto-builds voxpip.skill on tag push

Credits

Built on claude-video by @bradautomates (MIT).


github.com/EzekielOgunkunle/voxpip · LICENSE

Ver o original no GitHub

Bom para

Relacionados