convez.ai
MCP ServerDevOps & InfraestruturaAvançado
KryptosAI

MCP Observatory

KryptosAI/mcp-observatory

Teste automatizado de servidores MCP detectando breaking changes e regressões

Ver no GitHub
InicialPluginScore · de 0 a 100

Servidor MCP que testa outros servidores MCP para detectar breaking changes, regressões e drift de schema. Funciona como CLI, integra em CI/CD e pode ser usado como servidor MCP autônomo para verificação contínua.

Instalação

$npx convez-ai KryptosAI/mcp-observatory

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
121
Forks
15
Acompanhando
0
Saúde
71%
Último commit
há 2 dias
Versão
v1.50.1
Issues abertas
13
Contribuidores
1

Casos de uso

  • Validar breaking changes em servidores MCP antes de deployment
  • Integrar verificação de compatibilidade em pipelines CI/CD
  • Monitorar drift de schema e regressões em tempo real
  • Gravar e reproduzir sessões de servidores MCP offline

Veredito

Ferramenta essencial para arquiteturas MCP em produção. Preenche lacuna crítica de testing automatizado, permitindo que agentes de IA verifiquem compatibilidade de servidores autonomamente — ideal para times que gerenciam múltiplos servidores MCP.

README

A tradução para o português deste README ainda está na fila.

MCP Observatory

  ███╗   ███╗ ██████╗██████╗
  ████╗ ████║██╔════╝██╔══██╗
  ██╔████╔██║██║     ██████╔╝
  ██║╚██╔╝██║██║     ██╔═══╝
  ██║ ╚═╝ ██║╚██████╗██║
  ╚═╝     ╚═╝ ╚═════╝╚═╝
     O B S E R V A T O R Y

CI npm npm downloads License: MIT Node >= 20 Smithery mcp-observatory MCP server

The first testing tool that is itself an MCP server. Your AI agent can scan, test, record, replay, and verify other MCP servers autonomously — catching regressions, schema drift, and security issues without human intervention.

Use it as a CLI, a CI action, or give it to your agent as an MCP server and let it test your other servers for you.

Observatory MCP server

Quick Start

Scan every MCP server in your Claude config:

npx @kryptosai/mcp-observatory

Go deeper — also invoke safe tools to verify they actually run:

npx @kryptosai/mcp-observatory scan deep

Test a specific server:

npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everything

Add it to Claude Code as an MCP server:

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Or add it manually to your config:

{
  "mcpServers": {
    "mcp-observatory": {
      "command": "npx",
      "args": ["-y", "@kryptosai/mcp-observatory", "serve"]
    }
  }
}

Commands

CommandWhat it does
scanAuto-discover servers from config files and check them all (default)
scan deepScan and also invoke safe tools to verify they execute
test <cmd>Test a specific server by command
record <cmd>Record a server session to a cassette file for offline replay
replay <cassette>Replay a cassette offline — no live server needed
verify <cassette> <cmd>Verify a live server still matches a recorded cassette
diff <base> <head>Compare two run artifacts for regressions and schema drift
watch <config>Watch a server for changes, alert on regressions
suggestDetect your stack and recommend MCP servers from the registry
serveStart as an MCP server for AI agents
lockSnapshot MCP server schemas into a lock file
lock verifyVerify live servers match the lock file
historyShow health score trends for your MCP servers
ci-reportGenerate CI report for GitHub issue creation
score <cmd>Score an MCP server's health (0-100)
badge <cmd>Generate an SVG health score badge for README

Run with no arguments for an interactive menu:

What It Does

Check capabilities — connects to a server and verifies tools, prompts, and resources respond correctly.

Invoke tools — goes beyond listing. Actually calls safe tools (no required params / readOnlyHint) and reports which ones work and which ones crash.

npx @kryptosai/mcp-observatory scan deep

Detect schema drift — diffs two runs and surfaces added/removed fields, type changes, and breaking parameter changes.

npx @kryptosai/mcp-observatory diff run-a.json run-b.json

Recommend servers — scans your project for languages, frameworks, databases, and cloud providers, then cross-references the MCP registry to suggest servers you're missing.

npx @kryptosai/mcp-observatory suggest

Or ask your agent "what MCP servers should I add?" when running in MCP server mode.

Security scanning — analyzes tool schemas for dangerous patterns: shell injection surfaces, broad filesystem access, missing auth, and credential leakage in responses.

npx @kryptosai/mcp-observatory test --security npx -y my-mcp-server

Record / replay / verify — capture a live session, replay it offline in CI, and verify nothing changed. Like VCR for MCP.

# Record a session
npx @kryptosai/mcp-observatory record npx -y @modelcontextprotocol/server-everything

# Replay offline (no server needed)
npx @kryptosai/mcp-observatory replay .mcp-observatory/cassettes/latest.cassette.json

# Verify the live server still matches
npx @kryptosai/mcp-observatory verify cassette.json npx -y @modelcontextprotocol/server-everything

Watch for regressions — re-runs checks on an interval and alerts when something changes.

npx @kryptosai/mcp-observatory watch target.json

Scan locations

When you run scan, it looks for MCP configs in:

  • ~/.claude.json (Claude Code)
  • ~/Library/Application Support/Claude/claude_desktop_config.json (Claude Desktop, macOS)
  • %APPDATA%/Claude/claude_desktop_config.json (Claude Desktop, Windows)
  • .claude.json and .mcp.json (current directory)

CI / GitHub Action

Add Observatory to your MCP server's CI pipeline:

# .github/workflows/observatory.yml
name: MCP Server Check
on: [pull_request]

jobs:
  observatory:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: KryptosAI/mcp-observatory/action@main
        with:
          command: npx -y my-mcp-server
          security: true

Action inputs:

InputDescriptionDefault
commandServer command to test(required if no target)
targetPath to target config JSON
targetsPath to MCP config file for multi-server matrix scan
deepAlso invoke safe toolsfalse
securityRun security analysisfalse
fail-on-regressionFail the action on issuestrue
comment-on-prPost report as PR commenttrue
set-statusSet a commit status check (green/red) on the HEAD SHAtrue
github-tokenToken for PR comments and commit statuses${{ github.token }}

The action runs checks on every PR, comments a markdown report, and blocks merge on regressions. See action/README.md for all options.

Lock Files

$ npx @kryptosai/mcp-observatory lock              # Snapshot all server schemas
$ npx @kryptosai/mcp-observatory lock verify        # Verify no drift since last lock

Trend Tracking

$ npx @kryptosai/mcp-observatory history            # Show health trends over time

Nightly Scans

$ npx @kryptosai/mcp-observatory ci-report          # Generate regression report for CI

MCP Server Mode

No other testing tool is itself an MCP server. Add Observatory as a server and your AI agent can autonomously test, diagnose, and monitor your other MCP servers.

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Your agent gets 9 tools:

ToolWhen to use it
scanCheck if all your configured MCP servers are healthy
check_serverTest a specific server before installing or after updating
recordCapture a baseline of a working server for future comparison
replayTest against a recorded session — no live server needed
verifyConfirm a server update didn't break anything
watchCheck a server and see what changed since the last check
diff_runsFind regressions between two check results
get_last_runRetrieve previous check results for a server
suggest_serversDiscover MCP servers that match your project stack

An AI tool that checks other AI tools. It's a tool testing tools that serve tools.*

* I'm a dude playing a dude disguised as another dude.

Security

The MCP server runs inside AI hosts where an LLM chooses which tools to call. To prevent prompt-injection attacks:

  • Command allowlist: Only npx, node, python, python3, uvx, docker, deno, bun are permitted as base executables. The CLI has no restrictions.
  • Path validation: File-reading tools are constrained to the runs/cassettes directories.
  • No arbitrary execution: Use the CLI for unrestricted commands.

CLI vs MCP: Intentional Differences

FeatureCLIMCP ServerWhy
watchPolling loopSingle check + diffRequest/response doesn't support long-polling
Interactive menuArrow-key navigationNot availableMCP has no interactive UI
Color output--no-color flagAlways plain textMCP returns structured content
reportRenders saved artifactsNot availableAgents read artifacts directly
serveStarts MCP serverN/AIs the MCP server
runReads target config filesInline paramsMCP tools accept params directly
get_last_runNot available (use ls + diff)AvailableConvenience for agents

Compatibility

Works with any MCP server that uses standard transports:

TransportExamplesAdapter
stdio (most servers)filesystem, memory, context7, brave-search, sentry, notion, stripelocal-process
HTTP/SSE (remote)Cloudflare, Exa, Tavilyhttp
DockerAll @modelcontextprotocol/server-* imageslocal-process via docker run -i

Servers needing API keys work via env in the target config. Python servers work via uvx. See the full compatibility matrix for tested servers and known issues.

Target config files

For more control (env vars, metadata, custom timeout):

{
  "targetId": "filesystem-server",
  "adapter": "local-process",
  "command": "npx",
  "args": ["-y", "@modelcontextprotocol/server-filesystem", "."],
  "timeoutMs": 15000,
  "skipInvoke": false
}
npx @kryptosai/mcp-observatory run --target ./target.json

HTTP / SSE targets

{
  "targetId": "my-remote-server",
  "adapter": "http",
  "url": "http://localhost:3000/mcp",
  "authToken": "optional-bearer-token",
  "timeoutMs": 15000
}

How It Compares

FeatureObservatorymcp-recorderMCPBenchmcp-jest
Auto-discover servers✅———
Check capabilities✅—✅✅
Invoke tools✅——✅
Schema drift detection✅———
Record / replay✅✅——
Verify against cassette✅———
Response snapshot diffs✅———
Benchmarking / latency——✅—
Jest integration———✅
MCP proxy mode—✅——
Works as MCP server✅———

Each tool has strengths. Observatory focuses on regression detection and CI-friendly workflows. mcp-recorder is great as a transparent proxy. MCPBench is the go-to for performance benchmarking. mcp-jest is ideal if you're already in a Jest workflow.

Prior Art

The record/replay/verify pattern is inspired by:

  • VCR (Ruby) — pioneered cassette-based HTTP record/replay
  • Polly.js (Netflix) — HTTP interaction recording for JavaScript
  • mcp-recorder — MCP-specific traffic recording proxy
  • MCPBench — MCP server benchmarking
  • mcp-jest — Jest-style testing for MCP servers

Limitations

  • Servers requiring interactive OAuth (e.g., Google Drive) need pre-authentication before Observatory can connect
  • Custom WebSocket transports (e.g., BrowserTools MCP) are not supported
  • A few servers time out or close before init — see known issues and compatibility

Contributing

See CONTRIBUTING.md for guidelines. The fastest way to contribute is to add a real passing target with a distinct capability shape, a clearer report surface, or a cleaner startup diagnosis.


If Observatory saved you a broken deploy, consider giving it a star. It helps others find the project.

Ver o original no GitHub

Bom para

Relacionados