convez.ai
SkillUtilidadesIntermediário
mathiaschu

Watch — Análise de Vídeos Localmente

mathiaschu/watch

Dê ao Claude visão para assistir vídeos e responder perguntas sobre o conteúdo

Ver no GitHub
BomPluginScore · de 0 a 100

Skill que permite ao Claude analisar vídeos de praticamente qualquer fonte (YouTube, Instagram, X, Vimeo e ~1800 sites suportados por yt-dlp) extraindo frames e transcrevendo áudio 100% localmente sem chaves de API. Processa o vídeo completo — visual e áudio — para respostas contextualizadas.

Instalação

$npx convez-ai mathiaschu/watch

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
159
Forks
23
Acompanhando
1
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
5
Contribuidores
1

Casos de uso

  • Análise de conteúdo de vídeos long-form (aulas, palestras, streams)
  • Extração de informações visuais de vídeos (o que acontece em determinado timestamp)
  • Transcrição e sumarização de vídeos sem usar APIs pagas
  • Pesquisa em fontes protegidas (Instagram, X, YouTube privado) com autenticação do navegador

Veredito

Ferramenta robusta e bem arquitetada para processamento de vídeos offline. Elimina custos de API ao usar transcrição local (mlx-whisper/Whisper) e cobre vasto catálogo de fontes via yt-dlp. Ideal para quem precisa análise visual + áudio sem depender de serviços externos, com suporte a fontes autenticadas.

README

A tradução para o português deste README ainda está na fila.

/watch — give Claude eyes for video

Let Claude watch a video from almost anywhere — and transcribe it 100% on your machine, no API key.

Paste a URL (YouTube, Instagram, X/Twitter, Vimeo, TikTok, Loom, and ~1800 more) or a local file, ask a question, and Claude downloads the video, extracts frames, pulls a timestamped transcript, and reads every frame as an image. By the time it answers, it has seen the video and heard the audio.

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?

This is a fork of bradautomates/claude-video. The big change: transcription runs locally via mlx-whisper (Apple Silicon) or openai-whisper (CPU) instead of a paid Whisper API. No keys, no config file, no audio ever leaves your machine. Also adds browser-cookie support so login-gated sources (Instagram, X) work.


Install

Claude Code (plugin):

/plugin marketplace add mathiaschu/claude-video
/plugin install watch@claude-video

Generic skill folder (Codex, etc.):

git clone https://github.com/mathiaschu/claude-video.git ~/.claude/skills/watch

Then run once to install dependencies (or let the first /watch do it):

python3 ~/.claude/skills/watch/scripts/setup.py

Windows: use python instead of python3 (the python3 alias is a Microsoft Store stub). ffmpeg + yt-dlp install via the winget commands the setup script prints; for transcription use pip install openai-whisper (mlx-whisper is Apple Silicon only).

Requirements

  • Python 3.9+
  • ffmpeg + yt-dlp — auto-installed via Homebrew on macOS; on Linux/Windows the installer prints the exact command.
  • A local Whisper engine (only needed for videos without captions):
    • mlx-whisper — pip3 install mlx-whisper — preferred on Apple Silicon, runs on the GPU/Neural Engine.
    • openai-whisper — pip3 install openai-whisper — CPU fallback, cross-platform.

There is no API key. Captions cover most public videos for free; when a video has no captions, the audio is transcribed locally.

How it works

  1. You paste a video and a question. A URL (anything yt-dlp supports) or a local path (.mp4, .mov, .mkv, .webm, …).
  2. yt-dlp downloads it into a temp working directory. Local files are probed in place, no download.
  3. ffmpeg extracts frames at an auto-scaled rate. Duration-aware budget — ≤30s gets ~30 frames, 1-3min ~60, 3-10min ~80, longer 100 sparsely. Hard ceilings: 2 fps, 100 frames. JPEGs at 512px wide (bump with --resolution 1024 to read on-screen text).
  4. The transcript comes from one of two places. First: yt-dlp pulls native captions (free, instant). Fallback: a mono 16 kHz audio clip is transcribed locally with mlx-whisper / openai-whisper.
  5. Frames + transcript are handed to Claude. It Reads each frame as an image and aligns them to the timestamped transcript.
  6. Claude answers grounded in what's on screen and in the audio — not the title, not a guess.
  7. Cleanup. The script prints its working directory; Claude removes it when you're done.

Login-gated sources (Instagram, X, private videos)

Public videos download with no auth. For sources that require a login — Instagram, X/Twitter, age-restricted or private/unlisted YouTube — /watch needs your own browser cookies so yt-dlp can authenticate as you. You don't have to set anything up in advance: just run /watch <url> normally, and if it fails with a login/private error, do this.

The easy way — borrow cookies from your browser

  1. Make sure you're logged into the site (e.g. Instagram) in a normal browser.
  2. Tell Claude which browser it is, or run it directly:
/watch https://www.instagram.com/reel/XXXX/ --cookies-from-browser chrome

Supported: chrome, firefox, safari, edge, brave, chromium, opera, vivaldi.

macOS gotchas (the usual culprits):

  • Chrome must be fully quit (Cmd-Q, not just the window) — it locks its cookie database while running.
  • A Keychain prompt may appear ("… wants to use confidential information stored in Chrome Safe Storage"). Click Always Allow — that's macOS letting the tool decrypt Chrome's cookies on your own machine.
  • Safari requires giving your terminal / Claude Code Full Disk Access (System Settings → Privacy & Security → Full Disk Access).
  • When in doubt, Firefox tends to "just work" without closing it.

The reliable way — export a cookies.txt

If browser extraction keeps fighting you, export the cookies manually (works everywhere):

  1. Install a cookies exporter: "Get cookies.txt LOCALLY" (Chrome/Edge) or "cookies.txt" (Firefox) — both open-source.
  2. Open and log into the site (e.g. instagram.com).
  3. Click the extension → Export → save it, e.g. ~/Downloads/cookies.txt.
  4. Run:
/watch https://www.instagram.com/reel/XXXX/ --cookies ~/Downloads/cookies.txt

Privacy

Your cookies are read live from your own machine and piped straight into yt-dlp. The skill never copies, stores, logs, or transmits them. If you exported a cookies.txt, it stays on your disk — delete it whenever you want.

Useful flags

FlagWhat it does
--start T / --end TFocus on a section (SS, MM:SS, or HH:MM:SS); denser frame rate
--max-frames NLower the frame cap for a tighter token budget
--resolution WFrame width in px (default 512; 1024 to read on-screen text)
--fps FOverride auto-fps (clamped to 2 fps)
--cookies-from-browser BRead cookies from a local browser for login-gated sources
--cookies FILEUse a Netscape-format cookies.txt instead
--whisper mlx|openai-whisperForce a specific local engine
--no-whisperSkip transcription entirely (frames only)

Privacy

  • No API key, no account, no telemetry. There is no .env, no config file, no secret of any kind.
  • Audio never leaves your machine — transcription is fully on-device.
  • The only outbound network traffic is yt-dlp fetching the video/captions from the source URL, and a one-time ~1.5 GB Whisper model download from Hugging Face (cached afterwards).

Credits

Original skill: bradautomates/claude-video (MIT). This fork swaps the Whisper API for local transcription and adds browser-cookie support. Licensed MIT — see LICENSE.

Ver o original no GitHub

Bom para

Relacionados