convez.ai
MCP Server
TapeshN

qulib

TapeshN/qulib

Ver no GitHub
BomPluginScore · de 0 a 100

Honest QA gap analysis for deployed web apps. Auth-aware scanning, real accessibility checks, and an MCP server for AI agents.

Instalação

$npx convez-ai TapeshN/qulib

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
2
Forks
0
Acompanhando
0
Saúde
100%
Último commit
há 6 dias
Versão
v0.14.0
Issues abertas
14
Contribuidores
2

README

A tradução para o português deste README ainda está na fila.

Qulib

Release confidence for deployed web apps — the one question: should we ship?

Qulib fuses live-app quality evidence, automation maturity, and API coverage into a single scored verdict: ship / caution / hold / block. It prefers honest uncertainty over fake confidence: if auth blocks the crawl, coverage is thin, or data is incomplete, the report says so.

Design line: AI should explore unknown gaps; deterministic checks (crawl, axe, links, console) should scale. Cost Intelligence tracks LLM usage so repeated reasoning can graduate into checks you own in CI.

On npm: @qulib/core (engine + CLI qulib) and @qulib/mcp (MCP server for AI agents).

npm @qulib/core npm @qulib/mcp License: MIT


What Qulib does

  • Crawls deployed apps (anonymous or authenticated via Playwright)
  • Runs axe-core accessibility checks (WCAG 2 A/AA)
  • Detects broken links, console errors, navigation failures
  • Computes release confidence (0–100) with a coverage floor when too few pages were scanned
  • Emits JSON and Markdown reports (or ephemeral JSON on stdout)
  • Auth-aware: optional detect-auth, explore-auth, form-login, and storage-state flows
  • Cost Intelligence (optional block on gap analysis): token usage, budget warnings vs per-call output ceiling, prompt fingerprints, maturity hints, conversion recommendations

Packages

PackagePurpose
@qulib/coreAnalyzer engine and CLI (qulib)
@qulib/mcpMCP server exposing Qulib to AI clients

Quick start (CLI)

Release confidence — the flagship command:

npx @qulib/core confidence --url https://example.com

Returns a verdict (ship / caution / hold / block) with a 0–100 score, top risks, and recommended next checks.

Add --repo to also score test-automation maturity and API coverage:

npx @qulib/core confidence --url https://example.com --repo .

Analyze (full gap report):

npx @qulib/core analyze --url https://example.com

Scaffold a test suite:

npx @qulib/core scaffold --url https://example.com --framework cypress-e2e

Score automation maturity (repo only, no URL needed):

npx @qulib/core score-automation --repo /path/to/repo

From a clone (repo root):

npm run analyze -w @qulib/core -- --url https://example.com

Smoke (no disk writes):

npm run smoke

Cost doctor (after a normal analyze that wrote output/report.json):

npx @qulib/core cost doctor

Quick start (MCP)

{
  "mcpServers": {
    "qulib": {
      "command": "npx",
      "args": ["-y", "@qulib/mcp"]
    }
  }
}

Ask your agent:

Use Qulib to analyze https://example.com and tell me if it's ready to ship.

Default analyze_app responses are summary-first (top gaps, cost summary, next deterministic checks). Pass includeFullReport: true for the full gapAnalysis including all scenarios.


CI integration (GitHub Actions)

Gate your deploys on Qulib's honest agent-summary verdict (pass / warn / fail) with a drop-in action. There are two surfaces — pick whichever fits your CI:

Option A — composite action (drop into an existing job)

# .github/workflows/qa.yml
jobs:
  qa:
    runs-on: ubuntu-latest
    steps:
      - name: Qulib analyze gate
        uses: TapeshN/qulib/.github/actions/qulib-analyze@v1
        with:
          url: https://your-app.example.com
          fail-on: fail        # fail (default) | warn | never
          qulib-version: 0.9.0 # pin a version for reproducible CI

Option B — reusable workflow (whole job in one line)

# .github/workflows/qa.yml
jobs:
  qa:
    uses: TapeshN/qulib/.github/workflows/qulib-analyze.yml@v1
    with:
      url: https://your-app.example.com
      fail-on: warn
    # secrets:
    #   auth-storage-state: ${{ secrets.QULIB_STORAGE_STATE }}

How the gate decides

The action runs @qulib/core analyze --url <url> --agent-summary, which emits the stable agent-summary JSON. The CLI itself always exits 0 — the verdict lives in the gate field — so the action reads gate and turns it into a CI exit code:

gatefail-on: fail (default)fail-on: warnfail-on: never
pass✅ pass✅ pass✅ pass
warn✅ pass❌ fail✅ pass
fail❌ fail❌ fail✅ pass

A fail gate means a critical gap, a blocked scan, null/too-low confidence, or an auth-required surface that was never exercised — Qulib never silently passes an unevaluated deployment.

Every run uploads the agent-summary JSON as an artifact (qulib-agent-summary) and writes a job summary table with the gate, release confidence, top risks, and honesty notes.

Inputs (composite action)

InputDefaultDescription
url(required)Base URL of the deployed app to analyze.
fail-onfailGate policy: fail, warn, or never.
qulib-versionlatestnpm version/dist-tag of @qulib/core run via npx. Pin for reproducibility.
repo""Optional path to the app repo for repo-aware analysis.
config""Path to a qulib config file relative to the working dir.
auth-storage-state""Path to a Playwright storage-state JSON for authenticated scans (write a secret to a file first — never inline it).
extra-args""Extra raw flags appended to qulib analyze (advanced).
node-version20Node.js version to set up.
install-browsertrueInstall Playwright Chromium (Qulib crawls with Playwright).
output-pathqulib-agent-summary.jsonWhere to write the raw agent-summary JSON.

Outputs (composite action)

OutputDescription
gateThe verdict: pass | warn | fail.
release-confidenceRelease confidence 0–100, or n/a when null.
coverage-statusCoverage status enum (ok | thin | blocked-by-auth | …).
blockedtrue if the gate violated the fail-on policy (the job was failed).
summary-pathPath to the written agent-summary JSON artifact.

The composite action lives at .github/actions/qulib-analyze and the reusable workflow at .github/workflows/qulib-analyze.yml. Reference them at a stable tag (e.g. @v1) for reproducible CI.


Release confidence (short)

  • Score starts from 100 and is reduced by high / medium / low gaps (see gaps.ts).
  • If fewer than minPagesForConfidence pages were scanned, confidence is capped at 40 and a low-coverage warning is set—thin coverage must not read as “ready”.
  • auth-required early exit sets confidence 0 with no gap inventory: the deployment was not actually exercised.

Details: packages/core/README.md.


Confidence Layer (P3)

Qulib turns delivery signals into release confidence. The one question: ”Given everything we know right now, should we ship this?”

The Confidence Layer (v1, P3) fuses qulib's own evidence collectors into a single scored verdict:

VerdictMeaning
shipConfidence ≥ 80, no blockers, all required sources evaluated
cautionConfidence 30–79, or an unknown signal on a required source
holdConfidence < 30
blockA hard-blocking evidence item, or nothing evaluable

CLI:

qulib confidence --url https://example.com [--repo /path/to/repo] [--json]

MCP tool: qulib_score_confidence — composes analyze_app / qulib_score_automation / qulib_score_api into one fused verdict.

The 5 views (data model)

  1. Release Confidence — the fused score + verdict (available now; the computeReleaseConfidence output)
  2. Delivery Traffic — time-series of verdicts per subject (schema + diffConfidence helper; persistence P4)
  3. Inbox — human-judgment items: blockers, unknown signals, approvals needed (schema + deriveInbox; queue P4)
  4. Replay — provenance trace: how each score was computed and by which tool (buildReplay)
  5. Audit Trail — tamper-evident append-only ledger entry per verdict (toAuditEntry; sink P4)

Honesty over fake confidence: sources that are not_applicable, unknown, or null are excluded from the denominator but reported in contributions and honestyNotes. An auth wall or empty corpus forces verdict=block — qulib never silently passes an unevaluated surface.

Agent decisions are one evidence source. The reserved agent-evidence kind in EvidenceSourceKindSchema lets external agentic decisions feed into the same aggregator in a future release without touching the math.


Documentation


Contributing

Issues and PRs are welcome. See CONTRIBUTING.md. Follow the Code of Conduct.


License

MIT — see LICENSE

Ver o original no GitHub

Bom para

Relacionados