convez.ai
MCP ServerDevOps & InfraestruturaIntermediário
t4tarzan

TestForge MCP Server

t4tarzan/testforge

Análise de código em 22 dimensões — segurança, testes, performance e infraestrutura

Ver no GitHub
InicialPluginScore · de 0 a 100

Servidor MCP que analisa repositórios de código em 22 dimensões usando IA, detectando problemas de segurança, testes unitários, performance, acessibilidade e qualidade de infraestrutura. Funciona 100% offline com relatórios gerados em até 2 segundos, armazenando histórico localmente em SQLite.

Instalação

$npx convez-ai t4tarzan/testforge

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
2
Forks
2
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
5
Contribuidores
3

Casos de uso

  • Auditoria automatizada de segurança e qualidade de código antes de merge
  • Análise de conformidade de infraestrutura Kubernetes e manifests
  • Detecção de testes faltantes, vulnerabilidades de supply chain e dead code
  • Verificação de cobertura de escopo e alinhamento com stack tecnológico

Veredito

Ferramenta sólida e determinística que oferece análise profunda sem depender de chamadas LLM (opcionais em tier 2). Excelente para times que precisam de visibilidade rápida sobre qualidade, segurança e infraestrutura; suporta JavaScript/TypeScript e Python nativamente, com histórico persistido localmente.

README

A tradução para o português deste README ainda está na fila.

🧪 TestForge — AI CODE? Run TestForge!

22-dimension AI-powered testing. Drop any repo URL. Get a full report in under 2 seconds.

Website npm CI License


🚀 Try It

Web (managed): https://testforge.run — drop any public GitHub repo URL, get a 22-dimension report.

Local (MCP) — one command, no cloud, no sign-up:

npx -y @whitenoisenpm/testforge-mcp@latest
open http://localhost:33221

Dashboard accepts a local project path or a GitHub URL. The npm package ships better-sqlite3 prebuilt for macOS/Linux/Windows, so the SQLite engine that backs ~/.testforge/history.db installs with zero native-build steps. Your source code never leaves the machine.

🎬 30-second walkthrough — fully offline


📊 What We Analyze (22 dimensions)

CategoryDimensionsDetection method
Code QualitySecurity (SAST), Unit Tests, Load/Perf, AccessibilityBabel AST + taint tracking
InfrastructureKubernetes (manifests + Helm: securityContext, RBAC, probes, limits, NetworkPolicy)js-yaml + Helm-template stubbing
APIContract testing (OpenAPI cross-ref), Visual regressionAST + YAML parsing
AdvancedEdge cases, Property-based, Chaos, Mutation, PredictiveAST + cross-signal aggregation
StrategicVision & goals, Scope coverage, Stack analysisStrict dep-name sets + tsconfig parse
EnterpriseAgentic-scale, DORA, Supply chain (live OSV), N+1 queries, Dead code, License (SPDX), OWASPAST + lockfile→OSV.dev + node_modules walk

Tier-1 analyzers are deterministic — same input always produces the same output. No LLM calls, no Math.random(). Every dimension shipped substantive AST-based depth in v0.6.0 → v0.24.0 (16 deepening passes; see the mcp-server changelog). Tier 2 (v0.25.0) layers LLM-generated Vitest tests + sandbox execution on top — separate path, optional, BYOK on self-host. v0.26.0 closes the polyglot blind spot: Python (FastAPI / Flask / Django / pytest) is now native alongside JS/TS, and the dashboard surfaces a languageCoverage banner instead of pretending "0 endpoints" means "no endpoints" on repos written in languages we don't parse yet. Tier-2 now grounds generated tests in your real source (and imports & executes your real code where it safely can). Simulate (POST /simulate) boots the app and exercises the running system across opt-in lanes — load, chaos, agent, wired (real-code unit tests inside the booted image), and e2e (a Playwright crawl + LLM-authored user journeys). See the mcp-server README.

🆕 Recent work

The dated, authoritative history is the changelog (src/data/changelog.ts). The narrative — the arcs and the why — lives in the knowledge graph (docs/knowledge/, open as an Obsidian vault). Recent arcs (0.30 → 0.36.x):

  • Real simulation engine (0.30) — load / agent / chaos actually run against the booted app (autocannon + docker faults), not static guesses. See docs/knowledge/Simulation-Engine.md.
  • Kubernetes dimension (0.31) — the 22nd dimension: parses manifests + Helm and checks securityContext / RBAC / probes / limits / NetworkPolicy. Reports become grouped-by-dimension with per-dimension method + coverage + N/A.
  • No cry-wolf scoring (0.32 → 0.33) — diminishing-returns scoring replaces linear cliffs; 0/100 and null are gone, generated/vendored files and polyglot deps no longer trigger false flags. See docs/knowledge/Scoring.md.
  • Self-host UX (0.34 → 0.35) — setup wizard, local-AI (point Tier-2 at Ollama/LM Studio), in-dashboard Settings panel, Docker preflight, always-@latest install.
  • Managed BYOK + Tier-2 sandbox fixes (0.36.x) — hosted Tier-2 with your own (encrypted) OpenRouter key; the 0/0 ERRORED sandbox bug fixed; multi-arch + version-pinned runner images + build-locally fallback; full-detail Markdown report download. See docs/knowledge/Tier2-Sandbox.md.

🏗️ Architecture

┌──────────────────────────────────────────────────────────┐
│  Vercel — Frontend + 22 API routes (testforge.run)      │
│  React 19 + Vite + Tailwind + shadcn/ui                  │
│                                                          │
│  Auth:  GitHub OAuth → tf_session JWT cookie (httpOnly)  │
│  Edge:  CORS allowlist, X-Request-Id, Upstash rate-limit │
│  Data:  Neon Postgres via Drizzle ORM                    │
└────────────────────────────┬─────────────────────────────┘
                             │
                             │  /api/analyze passes through
                             ▼
┌──────────────────────────────────────────────────────────┐
│  Managed MCP — mcp.testforge.run (a VPS, behind nginx)   │
│  Fastify + TypeScript. Same analyzers as the npm package.│
│  Tier-2 sandbox via a locked-down docker socket-proxy.   │
│  /api/* proxies inject the run-secret + forward BYOK keys.│
└──────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────┐
│  Local MCP (npm package, default port 33221)             │
│  npx -y @whitenoisenpm/testforge-mcp@latest              │
│  SQLite at ~/.testforge/history.db                       │
│  🔒 No outbound calls except git clone + your AI provider│
└──────────────────────────────────────────────────────────┘

See docs/knowledge/Architecture.md for the full picture (endpoints, request flow, the shared analyzer core).


🔐 Security model

  • Auth: GitHub OAuth only. The callback mints a 30-day HS256 JWT and sets it as an httpOnly, secure, sameSite=Lax cookie named tf_session. Frontend never sees the token. Anonymous requests to user-scoped routes return 401, not seed data.
  • CORS: Allowlist enforced via Vary: Origin + origin reflection. Allowed: https://testforge.run, *.vercel.app preview deployments, and localhost dev ports. Anything else gets 403 "Origin not allowed".
  • Rate limit: Sliding window via Upstash Redis (@upstash/ratelimit). Default 60 req/min/IP. Falls back to in-memory with a console warning when Upstash isn't configured.
  • Request id: Every API response carries X-Request-Id (echoes Vercel's id when present). Logged with every line via the structured JSON logger — search Vercel logs by rid: for correlation.
  • Stripe webhook: Signature verified against the raw request body (bodyParser: false). Idempotent — duplicate event IDs return 200 {duplicate:true} via a stripe_events PK conflict.
  • Secrets posture: all credentials live in Vercel project env. .gitignore covers .env, .env*, .vercel, .secure/.

📦 Quick Start

Use the managed service

open https://testforge.run

Sign in with GitHub. Click "Managed", paste a repo URL.

Run the MCP locally (self-host)

# Start the server — dashboard at http://localhost:33221
npx -y @whitenoisenpm/testforge-mcp@latest

# Configure the AI provider for Tier-2 (OpenRouter or local Ollama/LM Studio)
npx -y @whitenoisenpm/testforge-mcp@latest setup

# Full env-var reference
npx -y @whitenoisenpm/testforge-mcp@latest --help

Tier-1 (22 dimensions) needs no config; Tier-2 needs an AI provider + Docker. No database to install — history auto-stores in SQLite at ~/.testforge/history.db. For IDE integration (Cursor / Claude / VS Code), add it as an MCP server — see docs/knowledge/Self-Host-and-BYOK.md.


🛠️ Local Development

git clone https://github.com/t4tarzan/testforge && cd testforge
npm install
npm run dev     # frontend + api on http://localhost:9999

# In another terminal:
cd mcp-server && npm install && npm run dev   # analyzer on :33221

Environment variables: copy .env.example to .env. The required ones to run anything authenticated are DATABASE_URL, SESSION_SECRET, GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET. See RUNBOOK.md for the full launch checklist.


💰 Pricing

PlanPriceTests/MonthReposKey Features
Free$051All 22 dimensions, public repos
Pro$29/mo10010Private repos, CI/CD webhooks, Slack/Discord
Enterprise$199/moUnlimitedUnlimitedSSO, SLA, dedicated support, custom integrations

🔌 API Reference

Full API docs at testforge.run/#/docs. Quick reference:

MethodEndpointAuthNotes
GET/api/healthnoneDB ping + version
GET/api/statusnoneAll-services rollup (parallel checks)
GET/api/badge?score=NnoneSVG badge for README embedding
GET/api/auth/callbacknoneOAuth start + callback; sets tf_session cookie
GET/api/auth/mecookieCurrent user; 401 if not signed in
POST/api/auth/logoutcookieClears tf_session
POST/api/analyzenoneProxies to Fly.io MCP; 502/504 on upstream failure (no fake fallback)
GET/api/projectscookieUser's projects
GET/api/historycookieUser's test runs
GET/api/reports/:idcookieOne report; 404 (not seed) when missing
POST/api/save-resultscookiePersist an analysis run
GET/POST/DELETE/api/keyscookieAPI key CRUD
GET/POST/api/gatecookiePlan quota — usage from real test_runs counts
GET/POST/api/stripecookie (POST)Checkout session
POST/api/stripe-webhookStripe signatureIdempotent via stripe_events table
POST/api/webhooknoneGitHub CI/CD webhook (notify/Slack/Discord)

All responses include X-Request-Id and rate-limit headers (X-RateLimit-Remaining, X-RateLimit-Reset).


🧪 Tests & CI

# Frontend + API E2E (Playwright)
npx playwright test
# Set BASE_URL=https://<preview>.vercel.app to run against a preview.

# Analyzer integration tests (vitest, against fixture projects)
cd mcp-server && npm test

CI runs on every PR via .github/workflows/ci.yml:

  • Lint + build — blocking. tsc -b && vite build.
  • Analyzer tests — blocking. vitest run against mcp-server/tests/fixtures/{vulnerable,clean}-app.
  • Playwright (preview URL) — blocking. Pulls the PR's Vercel preview URL, sends requests with the x-vercel-protection-bypass header so Vercel's SSO doesn't gate the run.

The bypass requires VERCEL_AUTOMATION_BYPASS_SECRET set as a GitHub Actions secret (mirroring Vercel's project-level "Protection Bypass for Automation"). See .github/workflows/ci.yml for the full setup steps.


📁 Project Structure

testforge/
├── api/                    # Vercel serverless functions
│   ├── _security.js        # withSecurity wrapper: CORS allowlist, rate limit, security headers, request-id, error catch
│   ├── _session.js         # jose-based HS256 JWT cookie, requireSession helper
│   ├── _env.js             # requireEnv() — typed env contract with clear missing-var errors
│   ├── _log.js             # JSON-line structured logger with per-request rid
│   ├── auth/               # callback, me, logout
│   ├── reports/[id].js     # single report; 404 (not seed) when missing
│   └── ...                 # analyze, projects, history, keys, gate, stripe, …
├── mcp-server/             # Fly.io / npm-published MCP server
│   ├── src/
│   │   ├── analyzers/      # 8 modules covering 22 dimensions
│   │   ├── local-db.ts     # SQLite via better-sqlite3
│   │   ├── mcp-server.ts   # /test, /quick-scan with SQLite persistence
│   │   └── index.ts        # Fastify app, port 33221
│   └── tests/
│       ├── analyzers.test.ts   # vitest against real fixtures
│       └── fixtures/{vulnerable,clean}-app/
├── src/                    # React 19 frontend (Vite)
│   ├── pages/              # 15 page components (HashRouter)
│   ├── context/AuthContext.tsx  # Hydrates from /api/auth/me cookie
│   └── db/schema.ts        # Drizzle ORM (8 tables, FK cascade)
├── drizzle/                # Generated migration baseline
├── scripts/
│   ├── migrate-to-v1.sql   # Idempotent one-shot to bring an existing Neon DB up to v1
│   ├── wrap-handlers.js    # Codemod that wraps every api/*.js with withSecurity
│   └── smoke.sh            # 10-assertion post-deploy smoke test
├── e2e/                    # Playwright E2E suite
├── RUNBOOK.md              # Pre-launch checklist + incident playbooks
└── LOG.md                  # Original build log

🏆 Built With

React 19 · TypeScript · Vite · Tailwind · shadcn/ui · Fastify · Neon Postgres · Drizzle ORM · Fly.io · Vercel · Stripe · Upstash Redis · jose · Playwright · Vitest · GSAP · Recharts · Framer Motion · better-sqlite3


📄 License

BUSL-1.1 — Free for non-production use and self-hosting. Commercial managed service requires a paid plan at testforge.run.

Ver o original no GitHub

Bom para

Relacionados