convez.ai
HookDevOps & InfraestruturaAvançado
jeremylongshore

Intent Audit Harness

jeremylongshore/intent-audit-harness

Containment de qualidade à prova de IA com hash-pinning e escape-scan

Ver no GitHub
BomPluginScore · de 0 a 100

Hook de pré-commit que aplica barreiras determinísticas contra degradação de qualidade induzida por IA. Fixa por hash arquivos de política de teste, escaneia tentativas de enfraquecimento de thresholds, e executa verificações de cobertura, arquitetura, viés e complexidade CRAP.

Instalação

$npx convez-ai jeremylongshore/intent-audit-harness

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
0
Saúde
100%
Último commit
há 4 dias
Versão
v2.0.0
Issues abertas
20
Contribuidores
3

Casos de uso

  • Bloquear propostas de IA que enfraqueçam testes ou limites de cobertura
  • Enforçar políticas de teste determinísticas e rastreáveis em CI/CD
  • Detectar mudanças silenciosas em arquivos de configuração críticos
  • Medir complexidade vs cobertura (CRAP) em múltiplas linguagens

Veredito

Ferramenta sofisticada para equipes maduras em qualidade de código que enfrentam pressão de automação IA. Requer disciplina de eng-owned policies, mas resolve problema real: mudanças silenciosas em thresholds críticos.

README

A tradução para o português deste README ainda está na fila.

@intentsolutions/audit-harness

npm License Provenance

Part of the Intent Eval Platform — the umbrella mapping the six repos that converge via a shared Evidence Bundle schema.

Deterministic test-enforcement toolkit. Companion to the audit-tests and implement-tests Claude Code skills — but usable standalone in any repo that wants hash-pinned, escape-scanned, AI-proof quality gates.

What it is

A small CLI wrapping 6 deterministic scripts:

CommandPurpose
audit-harness verifyVerify hash-pinned artifacts haven't changed since --init
audit-harness initPin the current state of engineer-owned policy files
audit-harness listShow pinned files
audit-harness escape-scan --stagedDetect AI attempts to lower test thresholds, delete tests, bypass architecture rules
audit-harness archRun language-appropriate architecture-rule checker (dependency-cruiser / import-linter / ArchUnit / deptrac / arch-go)
audit-harness biasCount common test-bias patterns
audit-harness gherkin-lintAdvisory Gherkin quality check
audit-harness crapCRAP (Complexity × Coverage) scorer — Python, Go, JS/TS, Rust

Install

Pick the install flavor that matches your repo's ecosystem — all three publish the same CLI surface.

Node / JS / TS (from npm):

pnpm add -D @intentsolutions/audit-harness
# or: npm install --save-dev @intentsolutions/audit-harness
# or: yarn add --dev @intentsolutions/audit-harness

Python (from PyPI):

pip install intent-audit-harness
# or inside a project venv:
python -m pip install intent-audit-harness

Rust (from crates.io):

cargo install intent-audit-harness

Any other language (Go, Ruby, PHP, Java, .NET, shell, etc.) — vendor the scripts:

curl -sSL https://raw.githubusercontent.com/jeremylongshore/intent-audit-harness/main/install.sh | bash

Quick usage

Pre-commit hook (.husky/pre-commit)

#!/usr/bin/env sh
pnpm exec audit-harness escape-scan --staged
pnpm exec audit-harness verify

CI workflow (.github/workflows/ci.yml)

  containment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: pnpm/action-setup@v5
      - uses: actions/setup-node@v6
        with: { node-version: '20', cache: 'pnpm' }
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec audit-harness verify
      - run: pnpm exec audit-harness escape-scan --range origin/main..HEAD

Engineer workflow — change a policy threshold

# 1. Edit tests/TESTING.md to change coverage.line from 80 to 75
# 2. Re-init to accept the change
pnpm exec audit-harness init
# 3. Commit the updated manifest alongside the policy change
git add tests/TESTING.md .harness-hash
git commit -m "chore(test): lower coverage floor to 75"

The containment model

The harness enforces this rule: policy changes must be conscious, not silent.

Engineer-owned files (tests/TESTING.md, features/*.feature, .dependency-cruiser.cjs, stryker.conf.json, etc.) are hashed into a manifest. Any diff that changes their content without a fresh audit-harness init is caught by pre-commit / CI and REFUSED.

AI agents remain useful (they can read policy, they can implement within constraints). What they can't do is silently weaken the constraints. That's the entire design.

See audit-tests/references/philosophy.md in the companion skill for the full rationale.

The 7-layer testing taxonomy

This harness sits inside a larger framework:

L7  Acceptance / RTM / Personas / Journeys     ← WHAT are we proving?
L6  E2E / BDD / Visual regression              ← User-level guarantees
L5  Perf / Security (SAST/DAST) / A11y / Chaos ← Non-functional
L4  Integration / Contract / Migration         ← Infrastructure wiring
L3  Unit + Coverage + Mutation + Arch + CRAP   ← Code-level correctness  ← audit-harness lives here
L2  Static analysis / Lint / Types / Secrets   ← Read-only scanning
L1  Git hooks / CI enforcement                 ← The cheapest gate       ← audit-harness enables this

The harness commands serve L1 (escape-scan in pre-commit + CI) and L3 (CRAP, architecture, bias, hash-pin).

Exit codes

Important for CI scripting:

ExitCommandMeaning
0anyClean
1escape-scanCHALLENGE — requires engineer-approved comment
2verifyHARNESS_TAMPERED — pinned file changed
2escape-scanREFUSE — pipeline halted
3verifyNo manifest (fresh repo, not an error)

Language support

Most scripts are language-agnostic (shell + regex). CRAP has per-language backends:

LanguageCRAPArchNotes
Pythonradon + coverage.pyimport-linterfull support
JS/TScomplexity-report + c8dependency-cruiserfull support
Gogocyclo + go test -coverarch-gofull support
Rustrust-code-analysis + tarpaulin(custom)coverage integration pending
Java/Kotlin—ArchUnitvia language-native tooling
.NET—ArchUnitNETvia language-native tooling
PHP—deptracvia language-native tooling

License

Apache 2.0 — see LICENSE and NOTICE.

Note: versions 0.x shipped under the MIT license. Starting with v1.0.0, the project is licensed under Apache 2.0. Existing 0.x releases on npm remain available under their original MIT terms; new releases (>= 1.0.0) are Apache 2.0.

Related

Versioning

SemVer. Breaking changes to the CLI surface bump major; new commands bump minor; bug fixes bump patch.

Contributing

This is infrastructure code. Changes need to be conservative. Before opening a PR:

  1. Read audit-tests/references/philosophy.md (in the companion skill) to understand the escape-grammar design
  2. Run bash scripts/escape-scan.sh --staged on your own diff — yes, the harness tests itself
  3. Add test cases if you're adding a new pattern to escape-scan or a new command to the CLI
Ver o original no GitHub

Bom para

Relacionados