EKTRO-MV
HorizonNowhere/EKTRO-MV
De uma frase para um clipe musical completo com IA
Ver no GitHubEngine open-source de IA que gera clipes musicais completos a partir de uma única frase. Integra geração de letras (Claude), síntese de voz (ACE-Step), criação de vídeos (Seedance), legendagem (Whisper) e edição final (Remotion) em um pipeline modular e extensível.
Instalação
npx convez-ai HorizonNowhere/EKTRO-MV1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Geração automatizada de videoclipes musicais para conteúdo de redes sociais
- Prototipagem rápida de conceitos visuais e musicais
- Pipeline de produção criativa com múltiplos provedores de IA intercambiáveis
Veredito

Solução ambiciosa e bem-arquitetada para automação criativa de videoclipes. Requer setup técnico robusto (Node 20+, ComfyUI, APIs pagas) e GPUs, mas oferece arquitetura modular genuína com suporte MCP/Hermes para integração em agentes.
README
A tradução para o português deste README ainda está na fila.
EKTRO-MV
One sentence → a music video.
Open-source AI music-video engine + CLI. Type one line, get a finished MV: LLM writes the song & shotlist, ACE-Step sings it, Seedance shoots it, Whisper captions it, Remotion cuts it.
Architecture
flowchart TD
IN["one sentence"] --> BRAIN["Brain · Anthropic Claude<br/>→ CreativeBrief (lyrics, shotlist, style)"]
BRAIN --> MUSIC["Music · ACE-Step / ComfyUI<br/>→ song.flac (vocals)"]
BRAIN --> VIDEO["Video · Seedance<br/>→ clip.mp4"]
MUSIC --> SUB["Subtitle · Whisper<br/>→ captions.srt"]
MUSIC --> COMP["Composite · Remotion"]
VIDEO --> COMP
SUB --> COMP
COMP --> OUT["ektro-mv.mp4<br/>H.264 / yuv420p / AAC"]
Every stage is a swappable provider behind a small interface in @ektro-mv/core
(BrainProvider, MusicProvider, VideoProvider, SubtitleProvider, CompositeProvider).
The defaults above are v1; add your own (Suno, Kling, local models…) by implementing the interface.
Hard Prerequisites
- Node 20+ and pnpm 10+
- ComfyUI with the ACE-Step node (GPU required for music generation)
- ffmpeg and ffprobe on
PATH(or setFFMPEG_PATH/FFPROBE_PATHenv vars) - ANTHROPIC_API_KEY — Claude API key
- ARK_API_KEY — Volcengine Ark key for Seedance video generation
Install
pnpm install
Env Setup
cp .env.example .env
# then fill in values: ANTHROPIC_API_KEY, ARK_API_KEY, COMFYUI_URL, EKTRO_WHISPER_INSTALL_DIR
Usage
# build the CLI first
pnpm --filter @ektro-mv/cli build
# generate a music video
node packages/cli/dist/bin.js "做一首赛博朋克 AI 觉醒神曲"
# or with output path
node packages/cli/dist/bin.js "make a cyberpunk anthem" --out mv.mp4 --workdir ./out
Subtitles (optional)
The Whisper subtitle stage is off by default (run with --skip-subtitles, or it is
skipped automatically when no subtitle provider is wired). To burn lyric captions into the MV:
# install the optional peer dependency (kept out of the default install — it's heavy)
pnpm add -w @remotion/install-whisper-cpp
# point at a writable install dir; the model downloads on first run
export EKTRO_WHISPER_INSTALL_DIR=~/.ektro-whisper # or set in .env
Then run without --skip-subtitles. Note: transcribing sung vocals is approximate, and
the first run downloads a whisper.cpp binary + model (~150 MB) from Hugging Face.
Development
# run all tests
pnpm test
# typecheck all packages
pnpm -r typecheck
# build all packages
pnpm -r build
Use from the HERMES agent
EKTRO-MV ships an MCP server (@ektro-mv/mcp) and a publishable Hermes skill
(skill/ektro-mv/SKILL.md) so any HERMES agent can drive the engine via one tool call.
1. Build the MCP server
pnpm --filter @ektro-mv/mcp build
2. Register with HERMES (one-time)
hermes mcp add ektro-mv --command "node /path/to/EKTRO-MV/mcp/ektro-mv-mcp/dist/bin.js"
Set the required env vars before starting the agent:
ANTHROPIC_API_KEY, ARK_API_KEY, COMFYUI_URL, EKTRO_WHISPER_INSTALL_DIR.
3. Call the tool
Once registered, ask HERMES to create a music video and it will call:
{ "tool": "ektro_mv_create", "arguments": { "prompt": "做一首赛博朋克 AI 觉醒神曲" } }
The tool returns the path to the rendered MP4.
4. Install the skill
Copy or symlink skill/ektro-mv/SKILL.md into your HERMES skills directory, or run:
hermes skills install ./skill/ektro-mv
The skill teaches HERMES when and how to invoke ektro_mv_create, and documents the
ektro-mv "…" CLI as a fallback when the MCP server is not available.
Package Structure
| Package | Role |
|---|---|
packages/core | Types, interfaces, config, Anthropic brain provider |
packages/providers | Seedance video, ACE-Step music, Whisper subtitle |
packages/composite | SRT parser, delivery gate, Remotion composite provider |
packages/cli | runMv orchestrator + ektro-mv bin |
apps/remotion | Remotion MusicVideo composition (looped video + song + captions) |
mcp/ektro-mv-mcp | MCP server exposing ektro_mv_create over stdio |
skill/ektro-mv | Publishable Hermes skill (SKILL.md + validation tests) |
License
MIT