Gauntlet - Harness Multi-Agente Gerador-Avaliador
vivekptnk/gauntlet
Audita, corrige e hardena repos em loops gated com juízes adversariais
Ver no GitHubFramework que implementa loops de auditoria e correção de código com separação rigorosa entre gerador e avaliador, gates de CI verbatim, e painéis de signoff adversariais. Garante que testes provem suas falhas e que código só passa quando múltiplos agentes independentes o aprovam.
Instalação
npx convez-ai vivekptnk/gauntlet1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Auditoria de segurança em repositórios com múltiplos agentes independentes
- Pipelines de CI/CD que exigem prova de que testes detectam regressões
- Hardening de código crítico com painéis de revisão gated e paralelos
- Orquestração de agentes que produzem e avaliam código sem conflito de interesse
Veredito

Gauntlet é uma metodologia sofisticada para validação de código via multi-agentes com separação entre gerador e juiz. Ideal para projetos que exigem garantias fortes, mas demanda setup complexo e entendimento profundo de CI/CD.
README
A tradução para o português deste README ainda está na fila.
Most AI coding loops pass tests and call it done. gauntlet doesn't trust that.
It separates the agent that produces work from the agents that judge it, runs every gate exactly as CI would, and demands proof that tests can actually fail - not just that they currently pass.
The gate is CI, verbatim. The evaluator is never the generator. Tests must prove they can fail.
What's Inside
| Mechanism | What It Does |
|---|---|
| Generator ≠ Evaluator | Producers never grade their own work. Every medium+ finding gets 2 independent skeptics prompted to REFUTE (a 3rd breaks a split). |
| The gate is CI, verbatim | No filtered subsets, no "just the new tests." Full suite at every major checkpoint - because a modified test class once hid the only red assertion. |
| Prove tests can fail | Every regression test must go red with the fix stashed. A test that passes on broken code is not coverage. |
| File ownership for parallel agents | Concurrent agents get disjoint file lists. No mid-flight commits. No partial-edit sweeps. |
Harness state in .harness/ | feature-list.json (goal tracking with verification contracts) + progress.md (append-only iteration log). Git-excluded, never stale. |
| Final signoff panel | Three read-only reviewers - MAINTAINER, ADOPTER, RELEASER - each empowered to block with "I would not ship this." |
| Six-phase sequence (A → F) | Recon → Audit → Fix → Build → Docs → Final loop. Structured, gated, repeatable. |
| Docs honesty | Every claim verified against current code. No invented numbers. Performance figures trace to committed docs or output you generated. |
┌─────────────┐
│ ◆ ◆ │
│ │ ≠ │ │
│ GEN EVAL │
│PRODUCER JUDGE│
└─────────────┘
Generator ≠ Evaluator Producers never grade their work. Every medium+ finding gets 2 adversarialskeptics; a 3rd breaks a split.
┌─────────────┐
│ stash │
│ fix → RED │
│ restore→GREEN│
│ PROVED FAIL│
└─────────────┘
Prove Tests Can Fail Stash the fix, confirm red. A test that passes on brokencode is not coverage.
┌─────────────┐
│ CI invoked │
│ VERBATIM │
│ full suite │
│ no filters │
└─────────────┘
Gate = CI, Verbatim Never a filtered subset. Full suite at every majorcheckpoint, no exceptions.
Installation
Claude Code one-liner (default)
curl -fsSL https://raw.githubusercontent.com/vivekptnk/gauntlet/main/install.sh | bash
The installer copies the skill bundle to ~/.claude/skills/gauntlet/ and verifies Claude Code can find it.
Codex one-liner
curl -fsSL https://raw.githubusercontent.com/vivekptnk/gauntlet/main/install.sh | bash -s -- --codex
The Codex installer target copies the same skill bundle to ${CODEX_HOME:-$HOME/.codex}/skills/gauntlet/ and verifies the frontmatter Codex uses to discover skills.
Git clone + install
git clone https://github.com/vivekptnk/gauntlet.git
cd gauntlet
./install.sh # Claude Code
./install.sh --codex # Codex
Manual copy
mkdir -p ~/.claude/skills/gauntlet
cp -R skills/gauntlet/. ~/.claude/skills/gauntlet/
mkdir -p "${CODEX_HOME:-$HOME/.codex}/skills/gauntlet"
cp -R skills/gauntlet/. "${CODEX_HOME:-$HOME/.codex}/skills/gauntlet/"
Zero-install: Bootstrap prompt
No skill infrastructure needed. Open BOOTSTRAP-PROMPT.md, fill the <ADD PROJECT GOALS ...> slot, and paste the whole thing into a fresh Claude Code or Codex session in your target repo. The orchestrator self-organizes from there.
Invoking the harness
Once installed, from any Claude Code session in your repo:
/gauntlet
Or with project-specific goals:
/gauntlet ship the auth module, harden the CI, and polish the README to portfolio grade
In Codex, ask for the skill by name from inside your repo:
Use gauntlet to ship the auth module, harden the CI, and polish the README to portfolio grade
The orchestrator reads your repo, writes .harness/, and proceeds through all six phases without asking permission between them.
For a non-mutating plan:
/gauntlet --dry-run ship the auth module
Or in Codex:
Use gauntlet --dry-run to plan the auth-module hardening pass
Dry run inspects the repo and returns the Phase A-F plan without editing files, creating .harness/, running gates, or committing.
To summarize an existing run:
Use gauntlet to show the harness status dashboard
How It Works
The six-phase sequence
| Phase | Name | What Happens |
|---|---|---|
| A | Recon + Baseline | Scope the repo, read roadmap and CI config, run the full build and test suite, record exact counts. Write .harness/feature-list.json and progress.md. |
| B | Audit Wave | 5-7 parallel auditors across dimensions (algorithmic correctness, concurrency/safety, API+doc accuracy, test quality, CI/tooling, portfolio presentation). Every medium+ finding goes through 2 adversarial verifiers (a 3rd on a split). Confirmed backlog saved to .harness/. |
| C | Fix Waves | Critical/high issues first (orchestrator may fix critical ones inline for precision), then file-disjoint fixer clusters + judges. Gate. Commit at verified checkpoints. |
| D | Build Wave | Remaining roadmap features, each with design-doc discipline. Acceptance bounds asserted in tests (assert, don't print). Judges verify. Gate. Commit. |
| E | Docs Wave | Writers + dual-lens judges running both lenses simultaneously: claim-by-claim accuracy AND visual/voice polish. Real output for visuals, honestly labeled. Gate. Commit. |
| F | Final Loop | Full suite + lint + release build + warnings-as-errors + signoff panel. Fix blockers, repeat F until zero blockers. Finalize .harness/, final commit. |
The gated loop
Each phase ends with a gate. The gate is the CI invocation, run verbatim from a clean checkout - never a filtered subset of tests, never just the new test classes. The orchestrator logs every iteration to progress.md (append-only) and does not move to the next phase until the gate is clean. The final signoff panel adds a human-quality check: three read-only reviewers, each with veto power, before declaring done.
Bundled roadmap features
- Stack-specific auditor profiles: Python, Swift, TypeScript, and Rust profiles ship in the skill bundle and are loaded only when Phase A detects the matching stack.
- Dry-run mode:
--dry-runor "plan only" produces the phase plan without mutating the repo. - Harness state viewer:
scripts/harness_state_view.pyrenders.harness/feature-list.json,progress.md, andaudit-backlog.mdas a compact Markdown or JSON dashboard.
Case Study
The methodology has run on real projects, including ProximaKit (public) and a private, in-progress MLX port (described generically below until it ships).
The clearest proof the gate discipline matters comes from that MLX port: a PyTorch-free port of a PyTorch imitation-learning policy for Apple Silicon, with numerical parity verified against the PyTorch reference. All local tests passed. The gate still caught four defect classes that ordinary local runs missed entirely.
What the gate caught that "all green" locally missed:
-
A
.gitignorerule silently excluded an entire source package from commits. Working tree: green. Clean checkout or CI: broken. Caught by clean-clone discipline - working-tree-green is not the same as checkout-green. -
A hidden transitive dependency (
pandas, pulled in viapyarrow.Table.to_pandas) that passed locally only because a dev-only extra had pulled pandas into the environment. Failed in a clean CI environment. Caught by CI plus a new subprocess test that blocks the import at the boundary. -
A YAML 1.1 float trap:
'1e-05'parses as a string in PyYAML, not a float. Would have silently poisoned the trainer's learning rate. Caught by a truthfulness judge diffing the generated config against the installed reference. -
A train-time-only model bug: the ported attention had no dropout - invisible to every passing numerical-parity test (all run in eval mode) but would corrupt training dynamics. Caught by an architecture-fidelity judge reading the port against the reference source side-by-side.
What the harness confirmed: numerical parity to the PyTorch reference was verified to ~1e-6 (end-to-end results ~430× inside the tolerance bound, comfortably within spec). The gate held the bar.
Full story: docs/CASE-STUDY.md
Lineage / Credits
gauntlet composes three published methodologies into a gated generator-evaluator loop:
- Anthropic - Building Effective AI Agents: the evaluator-optimizer and orchestrator-workers workflow patterns that underpin the generator/evaluator separation and the multi-phase loop structure.
- Anthropic - How we built our multi-agent research system: lead-agent/subagent orchestration and adversarial verification - the skeptical-verifier discipline (2+ independent REFUTE prompts per finding) comes directly from this.
- Geoffrey Huntley - Ralph: the external-state next-task → prompt → validate → log → repeat loop. gauntlet's
.harness/directory andprogress.mdappend-only log are the direct embodiment of this pattern.
gauntlet is an independent open-source methodology. It is not affiliated with or endorsed by Anthropic.
Roadmap
| Item | Status |
|---|---|
Core harness skill (/gauntlet in Claude Code, gauntlet in Codex) | Shipped |
| Bootstrap prompt (zero-install path) | Shipped |
| Claude Code install target | Shipped |
| Codex install target | Shipped |
| Stack-specific auditor profiles (Python, Swift, TypeScript, Rust) | Shipped |
--dry-run mode: produce the phase plan without executing | Shipped |
Harness state viewer (render .harness/ as a status dashboard) | Shipped |
License
MIT - use it, fork it, run it on anything.
Author: Vivek Pattanaik