FrexFrame: Extrator de Frames de Vídeo para IA
hrmnnwl/vid-frame-cli
Extração cirúrgica de frames de vídeo para análise com IA e debugging visual
Ver no GitHubCLI para extração inteligente de frames de vídeos do YouTube e arquivos locais, otimizado para análise com IA e debug de UI. Utiliza detecção de cenas e análise de keyframes para selecionar apenas momentos relevantes, integrando-se nativamente com Claude Code e APIs OpenAI.
Instalação
npx convez-ai hrmnnwl/vid-frame-cli1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Debug visual de interfaces e fluxos de UI
- Preparação de datasets para treinamento de modelos de visão computacional
- Análise de sessões gravadas de testes automatizados
Veredito

Ferramenta bem arquitetada para um problema real: transformar vídeos brutos em conjuntos de frames acionáveis para pipelines de IA. Bom valor para equipes que trabalham com testing visual e análise de comportamento de UI, mas o README carece de exemplos concretos de output final.
README
A tradução para o português deste README ainda está na fila.
FrexFrame: Intelligent Video Frame Extractor for Visual Debugging and AI Analysis
🚀 What is FrexFrame?
FrexFrame is a next-generation video frame extraction plugin designed specifically for Claude Code environments. It enables developers, AI researchers, and QA engineers to extract precise frames from video files for visual analysis, UI debugging, and model training datasets.
Think of FrexFrame as the surgical scalpel for video content — while standard tools dump hundreds of frames, FrexFrame intelligently selects only the most relevant moments. It acts as a cinematographer for your code, capturing the exact visual context your AI needs to understand, analyze, and debug your application's behavior.
"In a world where AI sees through text, FrexFrame gives it eyes for video."
🧠 Why FrexFrame Matters (The Problem We Solve)
Modern development workflows generate mountains of video data:
- Recorded UI test sessions
- Video walkthroughs of bug reproduction
- Time-lapse recordings of rendering processes
- Training data for computer vision models
Yet, most frame extraction tools are:
- Blunt instruments — extract everything, waste storage
- Context-blind — no integration with AI analysis pipelines
- Developer-unfriendly — require complex command-line gymnastics
FrexFrame bridges this gap. It is the Rosetta Stone between raw video files and AI-powered analysis, translating temporal visual data into structured, analyzable frames that Claude and other AI models can process.
🔬 Core Architecture (How It Works)
flowchart TB
A[Video Input File] --> B{Frame Extraction Engine}
B --> C[Scene Detection Module]
B --> D[Keyframe Analyzer]
B --> E[Timestamp Selector]
C --> F[AI Analysis Pipeline]
D --> F
E --> F
F --> G[Claude Code Integration]
F --> H[OpenAI API Bridge]
F --> I[Local Model Support]
G --> J[Visual Report]
H --> J
I --> J
J --> K[Debug Insights]
J --> L[UI Anomaly Detection]
J --> M[Training Dataset]
The architecture is designed as a layered onion — peel back complexity as needed:
- Layer 1: Simple timestamp extraction (seconds flat)
- Layer 2: Scene-aware keyframes (minutes)
- Layer 3: AI-powered relevance scoring (hours of video processed intelligently)
💻 Quick Start: Example Profile Configuration
Configure FrexFrame with a YAML profile to match your specific use case. Here's an example configuration that turns a recorded UI test session into actionable insights:
# config/frexframe-ui-debug.yaml
project_name: "ui_debug_session_2026"
video_source: "./recordings/login_flow.mp4"
output_dir: "./extracted_frames"
extraction:
mode: "intelligent"
max_frames: 12
min_scene_gap: 0.5 # seconds
ai_integration:
claude_api_key: ${CLAUDE_API_KEY}
openai_api_key: ${OPENAI_API_KEY}
analysis_prompt: |
Analyze these frames for UI inconsistencies,
layout shifts, and rendering errors.
debug_settings:
highlight_anomalies: true
generate_diff_report: true
output_format: ["json", "html", "markdown"]
os_compatibility:
- windows: optimal
- macos: optimal
- linux: optimal
🎮 Example Console Invocation
Run FrexFrame directly from your terminal with powerful command-line options:
# Basic extraction with default settings
frex extract ./demos/login-flow.mp4
# Intelligent extraction with AI analysis
frex analyze ./demos/shopping-cart.mp4 \
--max-frames 24 \
--mode intelligent \
--output ./results \
--ai-prompt "Find UI elements that change position between frames"
# Batch processing for regression testing
frex batch ./recordings/ \
--pattern "*.mp4" \
--config ./config/frexframe-ui-debug.yaml \
--parallel 4 \
--verbose
# Export to training dataset format
frex export ./results/ \
--format coco \
--labels ./labels.json \
--split train:val 80:20
💻 OS Compatibility Table
| Operating System | Version | Compatibility | Performance Notes |
|---|---|---|---|
| Windows 🪟 | 10/11 | ✅ Full Support | Native NTFS, GPU acceleration via CUDA |
| macOS 🍏 | Ventura+ | ✅ Full Support | Apple Silicon optimized, Metal API |
| Ubuntu 🐧 | 20.04+ | ✅ Full Support | Best for headless servers, Docker ready |
| Fedora 🐧 | 36+ | ✅ Full Support | Wayland compatible |
| Debian 🐧 | 11+ | ✅ Full Support | Minimal footprint, ideal for CI/CD |
| Arch Linux 🐧 | Rolling | ✅ Community Support | AUR package available |
| Android 📱 | 12+ | ⚠️ Partial | Limited to basic extraction |
| iOS 📱 | 16+ | ⚠️ Partial | Requires external storage |
✨ Feature List
Core Extraction Capabilities
- Intelligent Scene Detection — Automatically identifies scene changes and extracts only unique frames, eliminating duplicates and near-identical captures
- Timestamp Precision — Extract frames at exact millisecond intervals, perfect for synchronization tasks
- Batch Processing — Process thousands of videos concurrently with configurable parallelism
- Format Agnostic — Supports MP4, AVI, MOV, WebM, MKV, and 20+ other video formats
- Adaptive Quality — Automatically adjusts extraction resolution based on output requirements
AI Integration Features
- OpenAI API Integration — Send extracted frames directly to GPT-4 Vision for analysis, receiving structured JSON responses with detected anomalies
- Claude API Bridge — Native Claude Code plugin support with context-aware prompting that remembers previous analyses
- Custom Prompt Engineering — Define reusable analysis templates that guide AI interpretation of extracted frames
- Confidence Scoring — Each extracted frame receives a relevance score between 0-100, helping filter noise
Developer Experience
- Responsive UI — Real-time progress visualization with a terminal-based dashboard showing extraction status, frame previews, and analysis results
- Multilingual Support — Interface and documentation available in English, Spanish, French, German, Japanese, Chinese, and Arabic
- 24/7 Customer Support — Priority support for enterprise users with guaranteed response times under 2 hours
- Plugin Ecosystem — Extend functionality with custom pre-processing and post-processing plugins
Advanced Analytics
- UI Anomaly Detection — Automatically flag frames where UI elements shift, resize, or disappear unexpectedly
- Visual Regression Reports — Generate side-by-side comparisons of before/after frames with highlighted differences
- Metadata Extraction — Capture frame-level EXIF data, color histograms, and text content via OCR
- Export Flexibility — Output to JSON, CSV, COCO format, YOLO format, or custom schemas
🔑 SEO-Optimized Keywords
FrexFrame is positioned at the intersection of multiple high-search-volume topics:
- Video frame extraction tool
- AI visual debugging platform
- Claude Code video plugin
- OpenAI vision API frame analysis
- Automated UI testing visual feedback
- Video dataset preparation for machine learning
- Intelligent keyframe selection algorithm
- Developer debugging video tools
- Cross-platform frame extractor
- Visual regression testing automation
These keywords represent the core search queries that FrexFrame addresses, making it discoverable for developers, QA engineers, and AI researchers actively seeking solutions in the video analysis space.
🤖 AI API Integration Details
OpenAI API Integration
FrexFrame leverages OpenAI's Vision capabilities to analyze extracted frames:
# Conceptual integration example
import frexframe as ff
extractor = ff.FrameExtractor(api_key="sk-...", provider="openai")
frames = extractor.extract_analyze(
video="/path/to/video.mp4",
prompt="Identify any UI elements that overlap or render incorrectly",
model="gpt-4-vision-preview"
)
# Access structured results
for frame in frames:
print(frame.timestamp, frame.analysis.anomalies)
print(frame.confidence_score)
Claude API Integration
As a Claude Code plugin, FrexFrame provides seamless integration:
# Inside Claude Code environment
from frexframe.claude_plugin import FrexTool
tool = FrexTool()
result = tool.analyze_video(
video_path="debug_session.mp4",
context="Previous analysis showed button alignment issues",
max_tokens=4096
)
# Claude can now discuss specific frames by index
print(f"Frame 6 shows: {result.frames[6].description}")
🌟 Key Features Deep Dive
Responsive UI That Adapts to Your Workflow
FrexFrame's terminal user interface is not just a progress bar — it's a cockpit for video analysis. The interface dynamically adjusts based on:
- Terminal width and height (supports split panes)
- Current processing phase (extraction vs. analysis)
- User experience level (beginner vs. expert mode)
- Available screen real estate (responsive from 80x24 to 4K)
This ensures that whether you're running FrexFrame on a remote server via SSH or on a high-resolution desktop monitor, the experience remains consistent and informative.
Multilingual Support for Global Development Teams
Understanding that software development is a global endeavor, FrexFrame supports 7 languages at launch:
- English (default)
- Spanish — Interfaz completa en español
- French — Interface utilisateur en français
- German — Vollständige deutsche Benutzeroberfläche
- Japanese — 完全な日本語インターフェース
- Chinese — 完整的中文界面
- Arabic — واجهة كاملة باللغة العربية
Each language pack includes translated error messages, documentation, and CLI help text. Community contributions for additional languages are welcome.
24/7 Customer Support Architecture
Enterprise customers receive:
- SLA-backed response times: Critical issues resolved within 2 hours
- Priority queue: Support tickets jump the line
- Dedicated Slack channel: Direct chat with the development team
- Monthly strategy calls: Review extraction patterns and optimize workflows
- Custom integrations: Tailored plugins for specific CI/CD pipelines
⚠️ Disclaimer
Important Legal and Usage Notice
FrexFrame is provided as a developer tool for legitimate software development, debugging, and AI training purposes. Users are solely responsible for ensuring compliance with all applicable laws and regulations regarding:
-
Copyrighted Content: Do not extract frames from videos you do not own or have explicit permission to process. FrexFrame does not circumvent digital rights management (DRM) protections.
-
Data Privacy: When processing videos containing personal information (faces, license plates, personal documents), ensure compliance with GDPR, CCPA, and other privacy regulations.
-
AI Model Training: If using extracted frames to train AI models, verify that your use case does not violate the terms of service of the original video source or any third-party APIs.
-
Acceptable Use: FrexFrame is not designed or intended for surveillance, facial recognition without consent, or any application that violates human rights.
-
No Warranty: This software is provided "as is" without warranty of any kind, express or implied. The authors are not liable for any damages arising from its use.
-
API Terms: Usage of OpenAI or Claude APIs through FrexFrame is subject to the respective platform's terms of service and usage policies.
By using FrexFrame, you acknowledge these terms and accept full responsibility for how you use the extracted frames and analysis results.
📋 License
This project is licensed under the MIT License — a permissive license that allows you to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software.
Copyright (c) 2026
Permission is hereby granted, free of charge, to any person obtaining a copy of this software...
📦 Download and Installation
Installation Methods
Method 1: pip install
pip install frexframe
Method 2: Docker
docker pull frexframe/extractor:latest
Method 3: Source
git clone https://github.com/frexframe/frexframe.git
cd frexframe
pip install -r requirements.txt
python setup.py install
🌐 Community and Contributions
- Documentation: Full API reference available in the
/docsdirectory - Issue Tracker: Report bugs and suggest features on GitHub Issues
- Discussions: Join the community for tips, tricks, and best practices
- Contributions: Pull requests welcome! See CONTRIBUTING.md for guidelines
FrexFrame — Because every frame tells a story, and AI deserves to hear them all. 🎥🤖