LTX2-Vidgen Skill
patraxo/ltx2-vidgen-skill
Gere vídeos no Claude Code com seu próprio LTX-2.3 serverless
Ver no GitHubSkill para Claude Code que implanta seu próprio pipeline de geração de vídeos com LTX-2.3 (22B) em GPU serverless Modal. Suporta texto/imagem/interpolação/vídeo-para-vídeo com controles de pose/depth/canny, áudio sincronizado e múltiplos formatos (reel, YouTube, square) — tudo sob seu controle, a poucos centavos por clipe.
Instalação
npx convez-ai patraxo/ltx2-vidgen-skill1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Criar vídeos a partir de fotos com movimento natural
- Interpolar keyframes entre duas imagens
- Gerar vídeos verticais para reel/TikTok/Shorts
- Restylar e controlar vídeos existentes com LoRA
- Produção em lote com múltiplas variações
Veredito

Ferramenta robusta para quem precisa controlar totalmente sua pipeline de vídeo, com economia real vs. APIs comerciais. Exige configuração serverless (Modal) e conhecimento técnico, mas entrega qualidade profissional a fração do custo — ideal para produção em escala.
README
A tradução para o português deste README ainda está na fila.
A Claude Code skill that owns your AI video pipeline — drop a photo in Claude Code → get a video. It deploys your own optimized LTX-2.3 (22B) backend to your Modal serverless GPU and drives it: text-to-video, image-to-video, keyframe interpolation, video-to-video, and IC-LoRA (canny/depth/pose) control, with synced audio — a few cents a clip, your GPU, no per-clip API meter, no rate limit.
What this is
A Claude Code skill that owns the whole loop — it deploys your own LTX-2.3 (22B) video backend to your Modal account on first run, then drives it from Claude Code: drop a photo (or a prompt), ask for a video, get the .mp4. You never leave Claude Code; the GPU is yours; there's no SaaS in the middle.
- Modes: text-to-video, image-to-video, keyframe interpolation, video-to-video (retake/restyle), and IC-LoRA canny/depth/pose control — with synced audio.
- Batch:
--variations N(one prompt, N takes) and--prompts-file(many prompts) in one warm container. - Formats: reel / TikTok / Shorts (9:16), YouTube (16:9), square — native, no cropping.
- Optimized: resident pipeline + caches + torch.compile → ~1.96× faster, output bit-identical, bf16 throughout. Clips up to ~10 s; any resolution with sides ÷32.
Install
1. Install the skill (Claude Code, or any agent that reads SKILL.md):
npx skills add patraxo/ltx2-vidgen-skill # or: cp -R skills/ltx2-video ~/.claude/skills/
pip install modal && modal token new # Modal SDK + your account (free — $30/mo credits)
…or as a Claude Code plugin marketplace:
/plugin marketplace add patraxo/ltx2-vidgen-skill
/plugin install ltx2-vidgen@ltx2-vidgen-skill
2. Deploy your backend once — into your Modal account (downloads the LTX-2.3 weights + Gemma text encoder; public components only, no HuggingFace token):
git clone https://github.com/patraxo/ltx2-vidgen-skill && cd ltx2-vidgen-skill && ./deploy.sh
That's it. Now in Claude Code, drop a photo (or just describe a scene) and ask:
- "turn this photo into a video, subtle natural motion" — image → i2v
- "interpolate between these two frames" — two images → keyframe
- "restyle the middle of this clip" — a video → v2v
- "generate a neon city street, vertical reel" — prompt only → t2v
- "follow the edges of this clip" — a control render → IC-LoRA control
The skill validates the input, confirms cost (offers a cheap smoke first), calls your deployed app via modal.Cls.from_name (no endpoint, no auth, no secrets), and saves the .mp4 + a preview frame to ./video_out/. First clip cold-starts ~90 s; warm ~31 s.
PYTHONPATH=. uv run modal run deploy/ltx2_model.py::smoke_real --image-path pic.jpg # i2v
PYTHONPATH=. uv run modal run deploy/ltx2_model.py::run_modes # t2v + i2v + keyframe
PYTHONPATH=. uv run modal run deploy/ltx2_model.py::run_retake # v2v
PYTHONPATH=. uv run modal run deploy/ltx2_model.py::kf_real --image-a a.jpg --image-b b.jpg
PYTHONPATH=. uv run modal run tests/ship_verify.py # full verify
Modes
| Mode | Input | Underlying pipeline |
|---|---|---|
| text-to-video | prompt only | KeyframeInterpolation (0 keyframes) |
| image-to-video | 1 image + prompt | TI2VidTwoStages |
| keyframe interpolation | 2 images + prompt | KeyframeInterpolation |
| video-to-video (retake) | source video + window + prompt | RetakePipeline |
All four are exercised by run_modes / run_retake / kf_real and verified working (frame-inspected). The opt stack applies across every mode.
Performance
Runs on Modal serverless NVIDIA RTX PRO 6000 (Blackwell, 96 GB), bf16, billed per-second at $0.000842/s (~$3.03/hr) — so cost ≈ latency:
| Mode @ 768×1280 (9:16) | Warm latency | ~ $/clip |
|---|---|---|
| image / text / keyframe — 4 s (97 f) | ~31 s | ~2.6¢ |
| image / text / keyframe — 10 s (241 f) | ~80 s | ~6.7¢ |
| IC-LoRA control — 4 s | ~28 s | ~2.4¢ |
| video-to-video (retake) — 10 s | ~470 s | ~40¢ |
Cold start (first clip on a fresh container) ~90–200 s; idle scales to $0. Both 22B stage transformers stay resident — peak ~75 GB / 96 GB, verified zero OOM over a 10-generation run (<1 GB drift).
Optimization stack — bf16 throughout, cache paths bit-identical to the unoptimized run, ~1.96× net faster:
- Resident pre-fused pipeline + activation-aware resident cap + cross-resolution purge → no per-clip rebuild and no OOM when switching mode/resolution.
- First-block feature cache (~17%), CPU-pinned weight cache, text-embedding cache, torch.compile (persisted Inductor cache), tunable VAE-decode tiling.
- fp8 / SageAttention / flash-attn all tested and rejected — quality regressions or no sm_120 kernel. Speed comes from architecture, not from cheapening the math.
Costs — and what that buys you
New Modal accounts get $30/month in free credits. At ~2.6¢ per 4-second clip that's ~1,000 free clips/month — so you can explore a prompt 20 ways (--variations 20) and keep the one that lands, for pennies. It's your Modal account and bill (visible in your dashboard), no per-clip meter, no rate limit, idle = $0.
How it works
LTX-2.3 is a 22B video diffusion transformer. Serving it naively has two costs: a slow cold start, and a per-clip cost where the pipeline re-assembles + re-fuses model internals every request. This repo attacks both:
- Build once, stay resident. The fully-assembled, LoRA-fused transformer stays in GPU memory between clips (resolution-keyed, LRU-bounded) instead of rebuilt per request — the single biggest win when it fits (short/low-res: ~90 s → ~7 s).
- Both stages stay resident — safely. Two ~35 GB stage transformers + the (tiled) forward working set peak ~75 GB / 96 GB, so both stay GPU-resident at every supported resolution (measured; zero OOM over a 10-gen run, <1 GB drift). An activation-aware cap + a cross-resolution purge (drop residents from a prior, different-resolution request before the next forward) keep it safe — back-to-back mode/resolution switching never OOMs. (The earlier 94 GB OOMs were that cross-resolution accumulation, not the resident footprint.)
- CPU-pinned weight cache — weights pinned in host RAM, streamed to GPU, skipping disk reads. Bit-identical.
- First-Block-Cache — skips recomputing early transformer blocks (~17%, near-lossless).
- Embedding cache — the text encoder isn't re-run for a repeated prompt.
- Batching — 32 clips in one warm container ≈ 6.2× throughput.
- torch.compile + persisted Inductor cache — compiled once on the first cold container, restored from the volume on later cold starts (
max-autotunetested + rejected, slower here). - Blackwell-native attention — flash-attn has no sm_120 kernel yet, so this runs PyTorch SDPA (exact). fp8/SageAttention tested + rejected (noise / black frames). Speed comes from architecture, not from cheapening the math.
Helper modules live in utils/ and are mounted into the container.
Optimization flags (env vars, set in the image .env block)
| Variable | Default | Description |
|---|---|---|
LTX_PERSIST_PIPELINE | both | Keep stage transformers GPU-resident across requests. off/stage2/both. |
LTX_PERSIST_LRU_MAX | 2 | Upper bound on resident (stage, resolution) entries. The effective cap is computed per request (activation-aware, see below) and never exceeds this. |
LTX_VRAM_HEADROOM_GB | 40 | Free VRAM kept available before building a new resident transformer / loading the v2v/control pipeline. LRU residents are evicted to reach it, so a cross-mode/resolution build never OOMs the forward. |
LTX_VRAM_USABLE_GB | 91 | Usable VRAM for the activation-aware resident-cap math (96 GB card minus a safety margin). |
LTX_XFMR_GB | 35 | Assumed size of one stage transformer, used by the resident-cap math. |
LTX_REGISTRY | cpu_pinned | Weight cache: cpu_pinned (recommended) / gpu_resident / off. |
LTX_CACHE_TEXT_EMB | 1 | LRU cache on the text encoder output. |
LTX_SKIP_AUDIO | 0 | Per-request default for skipping audio decode (video pixels byte-identical). |
LTX_FP8 | 0 | Load official fp8 weights instead of bf16 (off = bf16 quality default). |
LTX_VAE_TILE_PX | 768 | VAE-decode spatial tile size (px, ≥64 & ÷32). Smaller → smaller decode peak (measured ~1 GB lever; decode is already well-tiled). Overlap blends seams. |
LTX_VAE_TILE_OVERLAP | 64 | Spatial tile overlap (px, ÷32, < tile). |
LTX_VAE_TEMPORAL_FRAMES | 80 | VAE-decode temporal chunk (frames, ≥16 & ÷8). |
LTX_VAE_TEMPORAL_OVERLAP | 24 | Temporal chunk overlap (frames, ÷8, < chunk). |
Layout
pyproject.toml # uv project (client-side dep: Modal SDK)
deploy.sh # one-command deploy
deploy/ltx2_model.py # the Modal app: t2v / i2v / keyframe / v2v + opt stack
deploy/utils/ # helper package (weight registries, FBCache, guiders)
skills/ltx2-video/ # Claude Code skill (SKILL.md + scripts/submit_video.py)
tests/ # smoke_test.py + ship_verify.py
assets/demo.gif
Acknowledgements & references
- LTX-Video (Lightricks) — the 22B model + upstream
ltx-core/ltx-pipelines. Paper: HaCohen et al., LTX-Video: Realtime Video Latent Diffusion — arXiv:2501.00103. - Modal — serverless GPU (RTX PRO 6000, per-second billing).
- Gemma-3 — text encoder.
Optimization techniques:
- Block-feature caching (the first-block cache here) — cf. BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching — arXiv:2509.13789; broader caching lineage e.g. SenCache (LTX-Video) — arXiv:2602.24208.
- VAE-decode spatial/temporal tiling, CPU-pinned weight cache, persisted
torch.compile(Inductor) cache, bf16-exact throughout (no fp8 / quantization).
By @patraxo. LTX-2.3 weights are distributed under Lightricks' license.