AutoResearch Skill
agusmdev/autoresearch-skill
Loop autônomo de otimização com medição rigorosa e versionamento Git
Ver no GitHubUma skill que transforma Claude em um agente de otimização autônomo, capaz de testar iterativamente melhorias em métricas mensuráveis (performance, bundle size, latência, loss, etc.). Funciona em um loop: tenta uma mudança, mede o resultado, mantém ganhos como commits Git e reverte regressões automaticamente, deixando um histórico auditável.
Instalação
npx convez-ai agusmdev/autoresearch-skill1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Otimizar tempo de execução de testes
- Reduzir tamanho de bundle de produção
- Tunar hiperparâmetros de modelos ML
- Melhorar latência e performance de parsers/algoritmos
Veredito

Skill robusto e bem arquitetado para quem precisa de otimizações iterativas rigorosas com rastreabilidade. Exige disciplina no setup de métricas e benchmarks, mas entrega excelente quando bem aplicado.
README
A tradução para o português deste README ainda está na fila.
autoresearch — an autonomous optimization-loop skill for Claude
Try an idea, measure it, keep what works, revert what doesn't, repeat.
autoresearch is a Claude Code / Claude skill that turns Claude into a disciplined optimization agent. Point it at a measurable target — test runtime, build/bundle size, latency, memory, ML training loss, a Lighthouse score — and it hill-climbs the metric through many small, measured changes: edit → benchmark → keep the wins (as commits) → revert the regressions → repeat, leaving an auditable trail the whole way.
It's a single git branch with many commits. Every kept experiment is one commit, so
git log is a clean record of what actually moved the number. Failed experiments are
reverted automatically (your run log is preserved). A confidence score tells you when a
"win" is real versus within benchmark noise.
Credit / inspiration
This skill is an independent reimplementation for Claude of pi-autoresearch by davebcn87 — an extension that brought the autonomous experiment loop to the pi coding agent. pi-autoresearch is in turn inspired by karpathy/autoresearch.
pi-autoresearch is MIT licensed. No source code was copied — pi-autoresearch is a TypeScript pi extension (custom tools + a live TUI widget), whereas this is a Claude skill (Markdown instructions + bundled shell/Python scripts) that reproduces the same workflow and semantics in Claude's very different runtime. All credit for the original idea, design, and the "try → measure → keep/revert → repeat, forever" framing goes to the pi-autoresearch and karpathy/autoresearch authors. If you use the pi coding agent, use the original — it's excellent.
Install
This skill is distributed for the skills.sh ecosystem
(npx skills, by Vercel). Requires Node.js 18+.
# Install into the current project (.claude/skills/)
npx skills add agusmdev/autoresearch-skill
# Or install globally for all projects (~/.claude/skills/)
npx skills add agusmdev/autoresearch-skill -g
git clone https://github.com/agusmdev/autoresearch-skill ~/.claude/skills/autoresearch
Claude Code picks up SKILL.md files in ~/.claude/skills/ (global) or
.claude/skills/ (project) on the next session.
After installing, the skill triggers automatically on requests like the ones below — or
you can invoke it explicitly with /autoresearch.
Usage
Just describe an iterative optimization with a metric. Examples that trigger it:
- "Optimize the runtime of our test suite in a loop — keep correctness, commit each win."
- "This JSON parser is too slow on big_input.json. Make it faster, but measure it properly."
- "Tune the hyperparameters in config.py to lower val_loss — it's noisy, so be rigorous."
- "Set up experiments to shrink the production bundle."
Claude will: confirm the goal/command/metric/scope → branch from a clean tree → write a benchmark and a session doc → take a baseline → then loop autonomously, committing improvements and reverting regressions, until it's exhausted promising ideas or you stop it. When it stops it reports baseline → best, % improvement, and what worked.
Running unattended
The loop is a resumable manual loop, not a background daemon — Claude drives it within
a turn. To run hands-off across context limits, wrap it with Claude Code's /loop or the
ralph-loop plugin, using a resume prompt like:
continue the autoresearch session in this repo: read autoresearch.md, run scripts/ar_status.py, then run the next experiment
All state lives in autoresearch.md + autoresearch.jsonl, so any fresh session can resume.
How it works
The skill is Markdown instructions plus three small scripts that handle the error-prone bookkeeping so every iteration is consistent and the result is auditable:
| Script | Role |
|---|---|
scripts/ar_run.sh | Runs the benchmark (./autoresearch.sh), times it, parses METRIC name=value lines, runs optional correctness checks. |
scripts/ar_log.py | Records a run: commits on keep, reverts code on discard/crash/checks_failed (preserving autoresearch.*), appends to autoresearch.jsonl, computes a confidence score. |
scripts/ar_status.py | Prints a dashboard from the log: baseline vs best, % gain, confidence, recent runs. |
Session files (created in your project during a run)
| File | Purpose |
|---|---|
autoresearch.md | Living session doc — objective, metrics, scope, off-limits, constraints, "what's been tried". A fresh agent resumes from this alone. |
autoresearch.sh | The benchmark. Prints METRIC name=value lines. Kept fast; editable mid-loop to add signal. |
autoresearch.jsonl | Append-only log: one config header per segment, one line per run. The machine-readable history. |
autoresearch.checks.sh | (optional) Correctness checks (tests/types/lint). Run after a passing benchmark; failure blocks keep. |
autoresearch.ideas.md | (optional) Backlog of promising-but-deferred ideas. |
These are deliberately named autoresearch.* so the auto-revert never deletes them.
Status semantics
- keep — primary metric improved →
git commit(the change becomes a commit). - discard — worse or unchanged → code reverted, no commit.
- crash — benchmark failed to run → reverted.
- checks_failed — metric was valid but correctness checks failed → reverted; can't be kept.
Confidence score
After 3+ measured runs, ar_log.py reports confidence = |best_improvement| / MAD
(Median Absolute Deviation of the metric — a robust noise floor): ≥2.0× likely real,
1.0–2.0× marginal, <1.0× within noise. It's advisory — it never auto-discards.
A regression never scores as "likely real", and the noise floor is estimated from kept
runs so exploratory failures don't bury a genuine win.
Full mechanics, the JSONL format, and resume/recovery steps are in
references/mechanics.md.
When NOT to use it
A heavyweight measured loop is the wrong tool for a trivial change. Skip it and just edit directly when: you already know the fix; the metric is trivial to eyeball and correctness is obvious; you'll try only one or two things and no reviewable history is needed; or there's no stable, repeatable way to measure the target. Reach for it when the work is genuinely iterative and measurement is the point. (The skill itself says this up front.)
What's in this repo
SKILL.md # the skill: setup, the loop, decision rules, stop conditions
scripts/
ar_run.sh # benchmark runner (timing + METRIC parsing + checks)
ar_log.py # keep/revert + jsonl log + confidence
ar_status.py # dashboard from the log
references/
mechanics.md # deep reference: semantics, jsonl format, confidence, resume
assets/
autoresearch.md.template # session-doc starter
autoresearch.sh.template # benchmark starter
Requirements: git, bash, python3 (stdlib only — no third-party packages).
License
MIT. Inspired by and modeled on pi-autoresearch (MIT) and karpathy/autoresearch.