Agent Factory CLI
PramodDutta/agent-factory-cli
Agentes de IA para análise, detecção e autocorreção de testes Playwright
Ver no GitHubConjunto de quatro agentes de IA (RCA, Flaky Detector, Self-Healing Locator, Triage) para automatizar análise e reparo de testes Playwright. Integra-se nativamente ao framework de testes com suporte a LLMs plugáveis (Groq, DeepSeek, Ollama) e orquestração por Claude Code, incluindo dashboard ao vivo para monitoramento.
Instalação
npx convez-ai PramodDutta/agent-factory-cli1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Classificar falhas de teste em categorias (produto, teste, ambiente, localizador)
- Detectar e rankear testes intermitentes (flaky) entre múltiplas execuções
- Propor e aplicar automaticamente localizadores resilientes no Page Object
- Gerar sumários prontos para CI/CD e criar tickets Jira com contexto completo
Veredito

Solução pronta e bem estruturada para QA automatizado em Playwright. O design modular (core + agentes específicos) e o dashboard integrado facilitam integração em frameworks existentes; requer configuração de LLM e compreensão de testes Playwright, mas entrega ROI real em redução de tempo de análise de falhas.
README
A tradução para o português deste README ainda está na fila.
🏭 agent-factory · AI agents for your Playwright framework
Four AI agents — RCA · Flaky Detector · Self-Healing Locator · Triage — that you drop into an existing Playwright framework as an
ai-agents/folder. Plus a live dashboard. Pluggable LLM (Groq · DeepSeek · Ollama). Orchestrated by Claude Code in production. Built for The Testing Academy QA Masterclass · Part 4.
What it does
| Agent | Job |
|---|---|
| 🔍 RCA | Classify every failure → product / test / env / locator + confidence + fix. Drafts Jira bugs. |
| 🎲 Flaky | Diff N runs of the same commit. Rank intermittents. Name the cause. Recommend @flaky. |
| 🩹 Heal | Read the real DOM. Propose a resilient locator (role > testid > text > css). Patch the Page Object. Refuses to guess (<0.7). |
| 🎯 Triage | Orchestrator. Runs all three, posts one PR-ready summary. The command you wire into CI. |
A dashboard (ai-agents/dashboard/) shows each agent's config, prompt, last verdict, and a Run button — served at localhost:4321.
This repo is the finished reference
In the masterclass you build this inside your own framework (AdvancePlaywrightFramework1x) under an ai-agents/ folder, prompt by prompt. This repo is the same code, packaged standalone so you can:
- run it today (
./ai-agents/demo.sh), - compare against your own build,
- or
npm i -g agent-factory-cliand point it at any Playwright report.
Follow the prompts at https://qamasterclass.vercel.app/claudeCodeForQA-Part4.
Layout
agent-factory/
├── ai-agents/ ← drop THIS folder into your framework
│ ├── core/
│ │ ├── llm.ts one OpenAI-compatible client + askJson()
│ │ ├── loader.ts parse Playwright JSON + error-context.md
│ │ ├── artifacts.ts write .runs/<agent>.json for the dashboard
│ │ ├── types.ts
│ │ └── log.ts
│ ├── rca/ prompt.ts + rca.agent.ts + README.md
│ ├── flaky/ prompt.ts + flaky.agent.ts + README.md
│ ├── heal/ prompt.ts + heal.agent.ts + README.md
│ ├── dashboard/ index.html + registry.json
│ ├── cli.ts dispatch rca|flaky|heal|triage|dashboard
│ └── demo.sh
├── bin/agent-factory-cli.js ts-node shim (global install)
├── examples/fixtures/ results.json + error-context.md + runs/*.json
└── .claude/ production · Claude Code subagents
├── agents/{rca,flaky,heal}.md
└── skills/triage/SKILL.md
Quickstart (standalone)
git clone https://github.com/PramodDutta/agent-factory-cli
cd agent-factory-cli && npm install
./ai-agents/demo.sh # LLM_MODE=mock · runs offline · opens dashboard
Or global:
npm i -g agent-factory-cli
agent-factory-cli triage ./test-results/results.json ./runs
Inside your own framework
cd AdvancePlaywrightFramework1x
# (build ai-agents/ via the Part-4 prompts, or copy this repo's ai-agents/ folder)
cp ai-agents/.env.example ai-agents/.env # set LLM_BASE_URL / LLM_API_KEY / LLM_MODEL
npm run ai:triage # rca → heal → flaky → summary
npm run ai:dashboard # http://localhost:4321
npm scripts to add to the framework's package.json:
{
"ai:rca": "ts-node ai-agents/cli.ts rca test-results/results.json",
"ai:flaky": "ts-node ai-agents/cli.ts flaky ./runs",
"ai:heal": "ts-node ai-agents/cli.ts heal --report test-results/results.json",
"ai:triage": "ts-node ai-agents/cli.ts triage test-results/results.json ./runs",
"ai:dashboard": "ts-node ai-agents/cli.ts dashboard"
}
Configure the LLM
Any OpenAI-compatible endpoint. cp ai-agents/.env.example ai-agents/.env:
# Groq · recommended free tier
LLM_MODE=live
LLM_BASE_URL=https://api.groq.com/openai/v1
LLM_API_KEY=gsk_xxx
LLM_MODEL=llama-3.3-70b-versatile
# Or DeepSeek → LLM_BASE_URL=https://api.deepseek.com · LLM_MODEL=deepseek-chat
# Or Ollama → LLM_BASE_URL=http://localhost:11434/v1 · LLM_MODEL=qwen2.5-coder:7b
LLM_MODE=mock → deterministic offline output (demos, tests).
Two paths
| Path | When | What it does |
|---|---|---|
CLI (ai-agents/) | Local · CI · cron | Prints verdicts. Drafts Jira. Patches Page Objects with --apply. |
Claude Code (.claude/) | Production | Calls Atlassian MCP for real Jira (JQL-dedup). Edits the real Page Object. Re-runs the spec to verify. |
Same prompts. The Claude Code variant adds the real side-effects.
@rca · @heal · /triage # inside Claude Code
Companion masterclass
Part 4 · Build the Agent Factory → https://qamasterclass.vercel.app/claudeCodeForQA-Part4 Reference framework → https://github.com/PramodDutta/AdvancePlaywrightFramework1x
License
MIT · © Pramod Dutta · The Testing Academy.