convez.ai
SkillAutomaçãoIntermediário
voidmatcha

e2e-skills — Geração, Revisão e Debug de Testes E2E

voidmatcha/e2e-skills

Gera, revisa e debugga testes Playwright/Cypress com análise de anti-patterns

Ver no GitHub
BomPluginScore · de 0 a 100

Suite de quatro skills para agentes Claude que automatiza o ciclo completo de testes end-to-end com Playwright e Cypress: gera testes E2E do zero com exploração em navegador vivo, revisa specs existentes contra 24 anti-patterns (P0/P1/P2), e debugga falhas capturando raízes de instabilidade. Detecta testes que passam mas não provam nada.

Instalação

$npx convez-ai voidmatcha/e2e-skills

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
27
Forks
3
Acompanhando
0
Saúde
57%
Último commit
há 2 dias
Versão
v3.6.0
Issues abertas
2
Contribuidores
1

Casos de uso

  • Gerar testes E2E automaticamente a partir de gaps de cobertura
  • Revisar specs existentes para false positives e test smells antes do CI
  • Classificar raízes de falhas de teste (timing, selector drift, auth, env)

Veredito

Ferramenta robusta para times que precisam de confiabilidade real em testes E2E. Resolve o problema crítico de testes que passam sem provar nada, com análise estática precisa e debugging rastreável. Recomendado para equipes exigentes em qualidade.

README

A tradução para o português deste README ainda está na fila.

e2e-skills — Playwright and Cypress E2E Test Generation, Review, and Debugging

E2E tests that always pass are worse than no tests — they give false confidence while real bugs slip through. e2e-skills is an AI agent testing toolkit for Playwright and Cypress: generate end-to-end tests, review existing specs for false positives and test smells, debug flaky E2E failures, and turn noisy CI reports into root-cause fixes. It runs as an Agent Skills bundle for Claude Code and Codex (and other AGENTS.md-compatible runtimes via the skills CLI) by @voidmatcha, catching what CI misses: tests that pass but prove nothing, and failures that are hard to trace.

Four complementary skills cover the full E2E testing lifecycle, from Playwright test generation to Cypress test review and failure debugging:

  1. playwright-test-generator — generates Playwright E2E tests from scratch, from coverage gap analysis to passing, reviewed tests
  2. e2e-reviewer — static analysis of existing Playwright and Cypress specs; flags 24 anti-patterns (P0 silent always-pass, P1 poor diagnostics, P2 maintenance) that can make tests pass CI while missing real regressions
  3. playwright-debugger — diagnoses failures from playwright-report/ and classifies root causes (flaky timing, selector drift, auth, environment mismatch, and more)
  4. cypress-debugger — same for Cypress report files

Contents

Install

# Recommended — install for Claude Code + Codex (most common)
npx skills add voidmatcha/e2e-skills --skill '*' -g -a claude-code -a codex

# Install everywhere — every agent the `skills` CLI supports
npx skills add voidmatcha/e2e-skills --skill '*' -g --agent '*'

# Claude Code plugin marketplace
/plugin marketplace add voidmatcha/e2e-skills
/plugin install e2e-skills@voidmatcha

# Manual clone (Claude Code)
git clone https://github.com/voidmatcha/e2e-skills.git ~/.claude/skills/e2e-skills

Codex users: install via the npx skills add route above (-a codex drops the bundle into ~/.codex/skills/, where Codex auto-discovers it).

Quick Example

You: Review my Playwright tests in apps/viewer/src/test/

e2e-reviewer:

  [P0] settings.spec.ts:88, 99 — #4h One-shot URL read
    expect(page.url()).toEqual(`${baseURL}/${id}-public`);   // sync read, no auto-retry
    → fix: await expect(page).toHaveURL(`${baseURL}/${id}-public`);
    (also removes redundant `await page.waitForTimeout(1000)` above)

  [P0] fileUpload.spec.ts:67 — #16 Missing await on action
    page.getByRole('button', { name: 'Delete' }).click();   // fire-and-forget, races next line
    → fix: await page.getByRole('button', { name: 'Delete' }).click();

  Total: 3 P0 (2 #4h, 1 #16), 0 P1, 0 P2 in 24 spec files.
  P1/P2 candidates (not yet flagged as bugs): 20× positional .nth() selectors, 5× direct page.click(selector).

Real findings from a recent typebot.io scan — silent always-pass bugs your test suite was hiding.

Workflow

  1. Run playwright-test-generator → generate with approval → auto-reviewed by e2e-reviewer
  2. Generated tests fail → playwright-debugger invoked automatically after 3 fix attempts
  3. Existing tests: e2e-reviewer → fix → re-run
  4. Tests fail → playwright-debugger or cypress-debugger → fix → re-run

Standalone Scanner

./skills/e2e-reviewer/scripts/scan.sh path/to/tests

Three tiers run in priority order: (1) eslint-plugin-playwright / eslint-plugin-cypress — uses your local install if present, otherwise auto-downloads via npx --yes (set E2E_SMELL_NO_ESLINT_DOWNLOAD=1 to disable); (2) ast-grep Tree-sitter rules for FP-prone patterns — uses ast-grep / sg on PATH if present, otherwise auto-downloads via npx --yes @ast-grep/cli (set E2E_SMELL_NO_AST_GREP_DOWNLOAD=1 to disable); (3) bundled regex coverage for grep-detectable P0/P1/P2 patterns and gaps the lint plugins miss — Cypress cy.on('uncaught:exception', () => false) blanket suppression (#3b), {timeout:0}.should("not.exist") (#4g), and cross-framework heuristics. See docs/e2e-test-smells.md for the full P0/P1/P2 model. Use // JUSTIFIED: <reason> on (or in the comment block directly above) an intentional pattern to suppress it in the bundled scanner output; the eslint tier does not parse JUSTIFIED markers — pair with an eslint-disable comment there if needed. The eslint tier also runs under a hang watchdog (E2E_SMELL_ESLINT_TIMEOUT_SECS, default 300s) and never blocks Tier 2/3 coverage when it fails.

The e2e-reviewer skill adds what no lint can reach: semantic checks (name-assertion mismatch, missing Then, YAGNI/zombie specs, POM consistency, auth setup analysis) and fix guidance with band-aid awareness. Run eslint-plugin-playwright / eslint-plugin-cypress as your every-commit baseline; invoke the skill for PR review, suspected silent-pass bugs, or before bulk fixes.

Proven in Open Source

Three real merged PRs, not synthetic examples:

RepositoryMerged PRWhat it fixed
Cal.comcalcom/cal.diy#28486False-passing Playwright assertions, no-op state checks, hard-coded waits → web-first assertions + condition waits
Storybookstorybookjs/storybook#34141Unawaited Playwright actions and discarded isVisible() calls that made E2E checks silently weak
Element Webelement-hq/element-web#32801Always-passing assertions, unawaited checks, toBeAttached() misuse, debugging leftovers

The skill was further iterated against 13 OSS Playwright/Cypress repos (1k+ stars) in a local testbed — zero GitHub side effects. The 4.4 cycle-count rule, 4.2 PR-culture cross-check, and Phase 2 retry-wrapper skip all came from observed agent behavior in those runs. See docs/case-studies.md for before/after lessons.

Skill 1: playwright-test-generator — Test Generation

Generates Playwright E2E tests from scratch for any project. Starts from coverage gap analysis, explores the live app via agent-browser tools, designs scenarios with your approval, and auto-reviews generated tests with e2e-reviewer.

When to Use

  • You have a page or feature with no E2E coverage
  • You want to bootstrap a test suite for an existing app
  • You need to quickly add tests before a release

Usage

Generate playwright tests
Generate playwright tests for the login page
Write e2e tests for the settings page
Add playwright coverage for checkout flow

Pipeline

  1. Detect environment — config, baseURL, test dir, POM structure, existing conventions doc
  2. Coverage gap analysis — user picks target (skipped when target given as argument)
  3. Live browser exploration — via agent-browser tools (no hallucinated selectors); accessible-name reality check for label-less inputs
  4. Scenario design + approval gate — shows plan and locator table before any code
  5. Code generation — POM + spec or flat spec, auto-detected from project conventions; writes must be route-stubbed (see Network Determinism in code-rules.md)
  6. Conventions & seed scaffolding (first run on a project) — appends a project-adapted E2E section to AGENTS.md and designates a seed spec, so future AI-generated tests (Claude Code, Codex, Playwright Agents) stay consistent
  7. YAGNI audit + e2e-reviewer — removes unused locators, catches P0 issues before first run
  8. TS compile + test run — 3 auto-fix attempts on failure (heal-by-intent locator re-resolution), then hands off to playwright-debugger

Skill 2: e2e-reviewer — Quality Review

Catches issues in E2E tests that pass CI but fail to catch real regressions.

When to Use

  • Your tests always pass but bugs still slip through to production
  • Tests pass CI but you suspect they miss real regressions
  • Your test suite is fragile — tests break on every UI change
  • You want to audit test quality before a release or code review
  • You're reviewing Playwright or Cypress specs

Usage

Review my E2E tests
Audit the spec files in tests/
Find weak tests in my test suite
My tests always pass but miss bugs
Tests pass CI but miss regressions
My tests are fragile and break on every UI change
We have coverage but bugs still slip through

24 Patterns Detected — Grouped by Severity

P0 — Must Fix (silent always-pass)

Tests pass when the feature is broken. No real verification is happening.

#PatternBeforeAfter
1Name-assertion mismatchName says "status" but only checks toBeVisible()Add assertion for status content, or rename to match actual check
2Missing ThenCancel action, verify text restored — but input still visible?Verify both restored state and dismissed state
3Error swallowingtry/catch in spec, .catch(() => {}) in POMLet errors fail; remove silent catch from POM methods
3bCypress uncaught:exception suppressioncy.on('uncaught:exception', () => false) blanket-swallows app errorsScope handler to specific known errors; re-throw unknown errors
4Always-passing assertiontoBeGreaterThanOrEqual(0); toBeAttached() with no comment; expect(await el.isVisible()).toBe(true) (one-shot); expect(await el.textContent()).toBe(x) (one-shot); expect(locator).toBeTruthy() (Locator always truthy); { timeout: 0 } on assertions (disables retry)toBeGreaterThan(0); toBeVisible(); web-first assertions with auto-retry
5Bypass patterns (5a P0, 5b P1)if (await el.isVisible()) { expect(...) }; { force: true } without commentAlways assert; move env checks to beforeEach; add // JUSTIFIED: to force:true
7Focused test leaktest.only(...) committed — CI runs one test, silently skips the restDelete .only; use --grep or --spec for local focus
8Missing assertionawait page.locator('.x'); (discarded); await el.isVisible(); (boolean thrown away)Add await expect(locator).toBeVisible() or delete the line
12Missing auth setupProtected-route spec navigates to /dashboard with no login/storageState/auth fixtureAdd beforeEach login, configure storageState, or use auth fixture — otherwise test passes against the login page
15Missing await on expect()expect(page.locator('.toast')).toBeVisible() returns an unobserved PromiseAdd await so the assertion actually runs
16Missing await on actionpage.locator('#submit').click() may not execute before the next lineAdd await so the action completes

P1 — Should Fix (poor diagnostics / wastes CI time)

Tests work but mislead developers, waste CI time, or set up future regressions.

#PatternBeforeAfter
6Raw DOM queriesdocument.querySelector in evaluate()Use framework locator/query APIs (locator / cy.get)
9Hard-coded sleepwaitForTimeout(2000) / cy.wait(2000) / waitForLoadState('networkidle')Rely on framework auto-wait; use condition-based waits
10Flaky test patternsitems.nth(2) without comment; test.describe.serial()Use data-testid or role selectors; replace serial with self-contained tests
13Inconsistent POM usagePOM imported but spec uses raw page.fill/page.click for POM-owned actionsRoute all interactions through the POM so UI changes update in one place
14Hardcoded credentialsloginPage.login('demo-admin', '<literal-password>') in test codeUse process.env.TEST_USER, Playwright config secrets, or test data fixtures
17Direct page.click(selector) APIpage.click('#submit') / page.fill('#input', 'text') skips the Locator layerUse page.locator(selector).click() for auto-wait and better error messages
18expect.soft() overuseAll assertions in a test are expect.soft() — test never fails earlyEnsure at least one hard expect() gates per test; use soft only for independent details
19Module-level mutable state in test codelet testNotebookSequence = 0; at column 0 in a test utility — collides across parallel workers and survives retriesDrop the counter; derive uniqueness from Date.now() + Math.random().toString(36).slice(2, 8), or move state into test.beforeEach
20Unmocked real-backend writesSignup/checkout spec submits real mutations — every CI run creates real accounts/ordersStub write/credential endpoints with page.route() / cy.intercept(); one designated real-backend smoke spec max
22Optimistic UI without call proofLike-toggle test asserts aria-pressed flip — UI updates optimistically, passes with the POST deletedPair UI assertion with page.waitForRequest() (armed before the click) or a route-hit flag

P2 — Nice to Fix (maintenance / robustness)

Weak but not wrong — addressed when refactoring.

#PatternBeforeAfter
11YAGNI + Zombie SpecsclickEdit() never called; empty wrapper class; single-use Util; entire spec duplicated by anotherDelete unused members; inline single-use Util methods; delete zombie spec files
21Manually-captured session-file dependencystorageState: 'auth/member.json' produced only by a manual capture script — absent on CI, silently expiresRegenerate session programmatically (API-login helper or setup project); manual files only as a cache with a programmatic fallback
23Fixture ignores render guardsLiked-tab fixture seeds liked: false; the card component return nulls every item — empty UI looks like infra flakeRead the item component's early returns/filters before seeding; seed fields to pass every guard for the view under test

References

Playwright best practices · Cypress best practices · Testing Library guiding principles


Skill 3: playwright-debugger — Playwright Failure Debugger

Diagnoses Playwright test failures from a playwright-report/ directory — whether failures happened locally or in CI. Classifies root causes and provides concrete fixes.

When to Use

  • You have a playwright-report/ directory (local or downloaded from CI) with failures to understand
  • Tests pass locally but fail in CI
  • You're dealing with flaky or intermittent test failures
  • You get TimeoutError or locator not found without a clear cause

Usage

Debug these failing tests
Why did these tests fail?
Tests pass locally but fail in CI

Note: Provide the report as a local path. Download CI artifacts manually from GitHub Actions and pass the directory path — automatic artifact fetching is not supported.

14 Root Cause Categories

#CategorySignals
F1Flaky / TimingTimeoutError, passes on retry
F2Selector Brokenlocator not found, strict mode violation
F3Network Dependencynet::ERR_*, unexpected API response
F4Assertion MismatchExpected X to equal Y, subject-inversion
F5Missing ThenAction completed but wrong state remains
F6Condition Branch MissingElement conditionally present, assertion always runs
F7Test Isolation FailurePasses alone, fails in suite
F8Environment MismatchCI vs local only; viewport, OS, timezone
F9Data DependencyMissing seed data, hardcoded IDs
F10Auth / SessionSession expired, role-based UI not rendered
F11Async Order AssumptionPromise.all order, parallel race
F12POM / Locator DriftDOM structure changed, POM not updated
F13Error Swallowing.catch(() => {}) hiding actual failure
F14Animation RaceElement visible but content not yet rendered

Debug Workflow

  1. Extract — parse results.json for failed tests, error messages, duration
  2. Classify — map each failure to F1–F14 using error signals (most failures resolved here)
  3. Trace — if still unclear, extract trace.zip and inspect step-by-step: failed actions, DOM snapshots, network errors, JS console errors
  4. Fix — concrete code suggestion per failure, P0/P1/P2 priority

Skill 4: cypress-debugger — Cypress Failure Debugger

Diagnoses Cypress test failures from mochawesome or JUnit report files. Classifies root causes and provides concrete fixes.

When to Use

  • You have a cypress/reports/ directory (local or downloaded from CI) with failures to understand
  • Cypress tests pass locally but fail in CI
  • You're dealing with flaky or intermittent Cypress failures
  • You get Timed out retrying or Expected to find element without a clear cause

Usage

Debug these failing Cypress tests
Why did these Cypress tests fail?
Analyze cypress/reports/
Cypress tests pass locally but fail in CI

14 Root Cause Categories

#CategorySignals
F1Flaky / TimingTimed out retrying, passes on retry
F2Selector BrokenExpected to find element, cy.get() failed
F3Network Dependencycy.intercept() not matched, XHR failed
F4Assertion Mismatchexpected X to equal Y, AssertionError
F5Missing ThenAction completed but wrong state remains
F6Condition Branch MissingElement conditionally present, assertion always runs
F7Test Isolation FailurePasses alone, fails in suite
F8Environment MismatchCI vs local only; baseUrl, viewport, OS
F9Data DependencyMissing seed data, cy.fixture() mismatch
F10Auth / Sessioncy.session() expired, role-based UI not rendered
F11Command Queue / Intercept Racecy.intercept registered after request fires; .then() chain order swap; parallel cy.request() race against an unfinished cy.visit()
F12Selector DriftDOM changed, custom command or POM selector not updated
F13Error Swallowingcy.on('uncaught:exception', () => false) hiding failures
F14Animation RaceElement visible but content not yet rendered

Debug Workflow

  1. Extract — parse mochawesome.json or JUnit XML for failed tests, error messages, duration
  2. Classify — map each failure to F1–F14 using error signals (most failures resolved here)
  3. Screenshot/Video — if still unclear, inspect cypress/screenshots/ and cypress/videos/
  4. Fix — concrete code suggestion per failure, P0/P1/P2 priority

License

Apache-2.0. See LICENSE.

Ver o original no GitHub

Bom para

Relacionados