convez.ai
AgenteAutomaçãoIntermediário
screenleon

QA Testing Rules

screenleon/qa-testing-rules

Regras QA estruturadas para agentes de IA evitarem testes fraco

Ver no GitHub
InicialPluginScore · de 0 a 100

Sistema de regras agnóstico de linguagem para testes de qualidade em agentes de IA. Define categorias de testes, identifica anti-padrões comuns e força revisão por mutação antes do deploy. Importável direto em CLAUDE.md ou .cursorrules para guiar agentes e engenheiros.

Instalação

$npx convez-ai screenleon/qa-testing-rules

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 4 semanas
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Estruturar testes em projetos com agentes de IA (Claude, Cursor, Gemini)
  • Implementar linting de anti-padrões de teste via Semgrep
  • Revisar cobertura de testes com categorização explícita

Veredito

Solução sólida para forcing discipline em testes gerados por IA. O approach de enumeração de categorias + catálogo anti-padrão + self-check por mutação é mais rigoroso que a maioria das abordagens ad-hoc. Requer investimento inicial em leitura, mas economiza debug posterior.

README

A tradução para o português deste README ainda está na fila.

qa-testing-rules

Language/framework-agnostic QA testing rules — a single source of truth that AI coding agents and human engineers can @import.

Why this exists

Without explicit guidance, LLM agents default to happy-path-only tests — pretty coverage numbers that catch no bugs. This repo forces three habits: (1) enumerate test categories before writing tests, (2) compare against an anti-pattern catalog, and (3) run a mutation self-review after writing.

Two-tier structure (important)

Tier 1 (agent hot path, always loaded, ~3k tokens — budget enforced in CI):

  • AGENT.md — the agent's only required entry point. Workflow + 12-category quick reference + red lines + self-checklist.

Tier 2 (reference, read only when needed):

FileWhen to read
PRINCIPLES.mdunsure how to judge an edge case
TEST-STRATEGY.mdchoosing test layers, designing CI / environments / coverage / flakiness / mutation-testing policy
TEST-CATEGORIES.mdstuck on concrete sub-cases for a category
GENERATIVE-TESTING.mdboundary / negative enumeration is endless (parser, serializer, money math) — property-based, fuzzing, model-based
ANTI-PATTERNS.mdreviewing / saw a smell and want to confirm it is an anti-pattern
EXAMPLES.mdneed good-vs-bad code comparisons
LLM-TESTING.mdthe SUT calls an LLM / output is non-deterministic
LEGACY-TESTING.mdadding tests to untested legacy code before changing it
CHEATSHEET.mdquick recall of layer choice, 12 categories, anti-pattern index

Usage (for agents)

Add to your project's CLAUDE.md / AGENTS.md / .cursorrules / GEMINI.md:

## Testing rules

When writing or reviewing tests, read:
https://github.com/screenleon/qa-testing-rules/blob/main/AGENT.md

Only consult the deeper reference files (PRINCIPLES / TEST-STRATEGY /
TEST-CATEGORIES / ANTI-PATTERNS / EXAMPLES / GENERATIVE-TESTING / LLM-TESTING / LEGACY-TESTING)
when AGENT.md explicitly points you to them, to keep token usage low.

Enforcement (machine-checkable subset)

Documentation does not stop violations; linters do. semgrep/qa-testing-rules.yml encodes the mechanically detectable anti-patterns (sleep-in-test, .only / .skip, weak assertions, bare toThrow(), os.Chdir / os.Setenv in tests, jest.retryTimes) as Semgrep rules for JS/TS, Go, and Python:

semgrep --config https://raw.githubusercontent.com/screenleon/qa-testing-rules/main/semgrep/qa-testing-rules.yml

Severity convention: ERROR = red-line violation (block CI), WARNING = fix or justify, INFO = needs human judgment.

Core stance

The purpose of testing is to find bugs, not to prove the program works. A test that does not fail when the implementation breaks might as well not exist.

Versioning

qa-testing-rules follows Semantic Versioning. Tags use vX.Y.Z (e.g. v1.1.0):

  • MAJOR: removing existing rules or changing the AGENT.md workflow step order (agents must relearn)
  • MINOR: new categories, examples, anti-patterns, or backward-compatible rule extensions
  • PATCH: wording fixes, formatting, changes that do not affect rule semantics

Other repos should pin a release tag or commit SHA, not the main branch. Replace vX.Y.Z with an existing release tag:

## Testing rules
When writing or reviewing tests, read:
https://github.com/screenleon/qa-testing-rules/blob/vX.Y.Z/AGENT.md
Only consult the deeper reference files when AGENT.md explicitly points you there.

Release process

  1. Make changes and let CI validate them (markdownlint, internal link check, AGENT.md token budget, changelog structure, semgrep rule validation — see .github/workflows/ci.yml). Local parity:
    npx markdownlint-cli2 '**/*.md'
    bash scripts/check-agent-token-budget.sh
    grep -q '^## \[Unreleased\]' CHANGELOG.md && echo "changelog ok"
    semgrep --validate --config semgrep/qa-testing-rules.yml
    
  2. Update CHANGELOG.md (move [Unreleased] → [vX.Y.Z] + date)
  3. git tag vX.Y.Z && git push origin vX.Y.Z

See CHANGELOG.md for history.

Acknowledgements

Condensed from:

Differentiation: the references above are integrated into an agent-executable step-by-step workflow (hot path under budget, deep details loaded on demand), plus an enforceable Semgrep rule pack.

License

MIT

Ver o original no GitHub

Bom para

Relacionados