convez.ai
InicialPluginScore · de 0 a 100

Turn your Claude subscription into a browser test agent

Instalação

$npx convez-ai shibukazu/ccqa

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
4
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 2 semanas
Versão
v1.61.0
Issues abertas
3
Contribuidores
2

README

A tradução para o português deste README ainda está na fila.

ccqa

Your Claude subscription already includes a QA engineer.

ccqa turns Claude Code into a browser test recorder. Write a spec in YAML, declare in the spec whether it should run deterministic or live, then ccqa run does the right thing per spec:

  • Deterministic (mode: deterministic, default): record once with ccqa record. Claude drives the browser, ccqa compiles every action into a test.spec.ts you can replay in CI under vitest — no LLM at run time. Cheapest and most stable.
  • Live (mode: live): no codegen. ccqa run sends each step to Claude every time, Claude drives agent-browser directly, judges pass/fail against the step's expected, and saves a before/after screenshot. More flexible for fragile UIs.

A single project mixes both: each spec.yaml picks its own mode, and ccqa run reads the field and dispatches. The HTML report covers both in one page.

No extra API key. Just claude.

日本語版 README

How it works

flowchart LR
    A["Write spec\n(spec.yaml + mode:)"] --> B{mode}
    B -- deterministic --> C["ccqa record\n(Claude → test.spec.ts)"]
    C --> D["ccqa run\n(vitest replay, no LLM)"]
    B -- live --> E["ccqa run\n(Claude drives every time,\nper-step pass/fail)"]

For deterministic specs, record invokes Claude Code with your spec, Claude drives the browser step by step, every action is recorded, and a vitest-compatible script is generated. run then replays it without involving an LLM.

For live specs, record is not needed. run directly sends each step to Claude, which drives the browser through agent-browser, judges whether the step's expected clause holds, and writes a PNG before and after each step. Useful when codegen is fragile (timing-dependent UIs, rich-text editors, dynamic selectors).

Install

pnpm add -D ccqa vitest agent-browser

Requires Node.js 20+. agent-browser is a peer dependency.

Quick start

1. Write a spec — by hand, or interactively with ccqa draft. Declare the mode in the spec itself.

# .ccqa/features/tasks/test-cases/create-and-complete/spec.yaml
title: Create a task and mark it complete
mode: deterministic   # or: live. Omit for deterministic (the default).

steps:
  - instruction: |
      Open ${APP_URL}/login. Fill in email and password, submit the form.
    expected: Redirected to /dashboard, user avatar visible in the header

  - instruction: |
      Click "New Task", fill in the title "Fix login bug", set priority to High, save.
    expected: Task appears in the task list with status "Open"

URLs live inside instruction strings — either verbatim or via ${ENV_VAR} references for environment-specific values.

2a. For mode: deterministic — record once, then replay

ccqa record tasks/create-and-complete   # Claude drives the browser; generates test.spec.ts
ccqa run tasks/create-and-complete      # vitest replays test.spec.ts; no LLM

2b. For mode: live — skip codegen, run directly

ccqa run tasks/create-and-complete      # Claude drives the browser every time

By default deterministic runs write step-boundary screenshots and metadata to ccqa-report/evidence/<feature>/<spec>/ so a reviewer can confirm a passing spec actually reached the states its expected clauses describe. Disable with --no-evidence.

In CI you can opt in to an HTML run report by passing --report — every failing spec gets a drift audit plus a root-cause call (TEST_DRIFT / SPEC_CHANGE / PRODUCT_BUG) using the branch's git diff as context, and the report lets a human grade those calls to measure their accuracy. Requires ANTHROPIC_API_KEY or a local Claude login for the analysis part. Opt out with --no-failure-analysis (which also implicitly skips the drift audit — the audit is rendered as evidence under the classification, so without the classification the cost has nowhere to land). Use --no-drift-audit to keep the classification but skip the audit. See Run report.

ccqa run tasks/create-and-complete --report --base origin/main
ccqa run --changed --report                    # only specs whose relatedPaths touch the diff

Features

FeatureDocs
Write specs interactively with ClaudeDraft
Reuse login and other shared step sequencesBlocks
Assertion helper functionsAssertions
Auto-fix failing testsAuto-fix
Detect spec/code drift in CIDrift
HTML run report with failure root-cause callsRun report
Inventory existing test coveragePerspectives
Architecture decision records (why it is built this way)ADR

Commands

ccqa draft [feature/spec]          Co-author a test spec with Claude
ccqa perspectives                  Inventory existing test coverage into .ccqa/perspectives.yaml
ccqa record <feature/spec>         (deterministic specs only) Trace browser actions + generate test.spec.ts
ccqa run [feature/spec]            Execute specs. Per spec, the spec.yaml `mode:` field selects deterministic
                                   (vitest replay) or live (Claude drives every time). One run can mix both;
                                   `--report` writes one unified HTML.
ccqa drift [feature/spec]          Standalone spec ↔ codebase static audit (for PR checks)

ccqa run flags:

  • --report [dir] — write a self-contained HTML run report (default dir: ccqa-report/)
  • --changed — restrict execution to specs whose relatedPaths intersect git diff <base>...HEAD. Mutually exclusive with an explicit spec id.
  • --base <ref> — base ref for the git diff (default: $GITHUB_BASE_REF, then origin/main)
  • --no-failure-analysis — skip the per-failure root-cause classification (also skips the drift audit, since the audit only shows under the classification)
  • --no-drift-audit — skip the spec ↔ code drift audit while keeping the classification
  • --no-evidence — (deterministic specs only) skip step-boundary PNG capture
  • --retry <n> — (live specs only) retry each failing step up to N more times
  • --format <fmt> — text (default), json (report.json), github (Actions annotations)
  • --out <dir> — (live specs only, single-spec invocations) override the per-run artifact directory

All Claude-driven commands accept -m, --model <name> (alias sonnet | opus | haiku, or a full model ID). The flag overrides CCQA_MODEL; when both are unset, the Claude Code CLI default is used. They also accept --language <bcp47> (e.g. ja, en) to set the language of human-readable output; the default auto follows the language of the spec/codebase. --cwd <path> works on record / run / drift so you can target a subpackage inside a monorepo from the repo root. Interactive commands authenticate via your local Claude Code login; commands that talk to Claude in CI (ccqa run --report, ccqa drift) additionally honor ANTHROPIC_API_KEY.

<feature/spec> is a 2-segment alias for the on-disk path .ccqa/features/<feature>/test-cases/<spec>/.

File structure

.ccqa/
  perspectives.yaml              # Inventory of existing coverage (machine-readable, canonical)
  perspectives.md                # Category index, regenerated from the YAML
  prompts/
    trace.user.md                # Project-specific guidance appended to `ccqa record` (trace phase)
    run-nd.user.md               # Project-specific guidance appended to `ccqa run` (live specs)
  blocks/
    login/
      spec.yaml                  # Reusable block (params + steps)
  features/
    tasks/
      perspectives.md            # Per-category detail tables (one per case)
      test-cases/
        create-and-complete/
          spec.yaml              # Test definition, with `mode: deterministic | live`
          actions.json           # (deterministic only) Recorded actions from `ccqa record`
          test.spec.ts           # (deterministic only) Generated vitest script
          runs/
            2026-06-14T10-00-00-000Z/  # (live only) one `ccqa run` invocation
              run.json                  # Machine-readable summary
              run.md                    # Human-readable per-step log
              steps/
                step-01.before.png      # Before-step screenshot
                step-01.after.png       # After-step screenshot
                step-01.log.txt         # Claude's full transcript for the step

Add .ccqa/features/*/test-cases/*/runs/ to .gitignore — these are per-run artefacts that should not be committed. Likewise ccqa-report*/.

Live specs (mode: live)

For specs declared mode: live in their spec.yaml, ccqa run skips codegen entirely: Claude executes each spec step against agent-browser directly, judges whether the step's expected outcome holds, and saves a PNG screenshot before and after every step. Use this mode when:

  • you want to validate a spec but don't yet need a replayable, recorded test
  • the codegen output for a spec is fragile (heavily timing-dependent UIs, rich-text editors, dynamic selectors)
  • you want a visual audit trail of what the page looked like at every step
# Run a single live spec
ccqa run tasks/create-and-complete

# Run every spec under a feature (mixes deterministic + live as declared)
ccqa run tasks

# Run every spec in the project, into a unified HTML report
ccqa run --report

# Retry each failing step up to 2 more times (live specs only)
ccqa run --retry 2 tasks/create-and-complete

Constraints on selectors / agent-browser subcommands that apply during ccqa record (no eval, no @ref, no bare-tag positional find, no chained agent-browser calls) are relaxed for live specs — Claude can use any subcommand and any selector style because there is no replay contract to honour.

Per-project guidance (.ccqa/prompts/run-nd.user.md)

ccqa's live-mode system prompt is deliberately product-agnostic. Anything specific to your project — staging URLs, login flow quirks, rich-editor types, common access-denied wording — belongs in .ccqa/prompts/run-nd.user.md. The file is read once per invocation and appended to the system prompt under a "Project-specific guidance" heading.

Keep it short. A page or two of focused notes beats a long handbook — Claude has the spec's expected text to work from, the file is for the non-obvious product knowledge that isn't in any single spec. Examples of what's useful here:

  • "the rich text editor is [contenteditable='true'] — use fill, not keystrokes"
  • "login redirects through an IDP service-selection screen; you can skip it by opening the destination URL directly"
  • "access-denied is signalled by a specific in-app message string — name it here so the model asserts on it"

Examples of what does not belong:

  • per-spec details (those belong in the spec's instruction / expected)
  • restating the STEP_RESULT contract (already in the system prompt)
  • copy-pasted style guidelines from trace.user.md (the relaxed-constraint mode doesn't need them)

The file is capped at 32 KiB; anything beyond that is truncated with a warning.

License

MIT

Ver o original no GitHub

Bom para

Relacionados