convez.ai
MCP ServerDados & AnalyticsAvançado
Keshetti-Nikhil

Data Catalog Lineage MCP Server

Keshetti-Nikhil/data-catalog-lineage-mcp-server

Análise de impacto de mudanças em colunas com linhagem de dados de alta precisão

Ver no GitHub
InicialPluginScore · de 0 a 100

Servidor MCP em TypeScript que expõe ferramentas de linhagem sobre um grafo de dependências de 400 mil ativos, com análise de impacto em nível de coluna. Inclui benchmark comparando quatro métodos de impacto, mostrando que linhagem em nível de coluna (F1 0.79) reduz alertas falsos em 8x versus análise transitiva completa.

Instalação

$npx convez-ai Keshetti-Nikhil/data-catalog-lineage-mcp-server

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Avaliar impacto de mudanças em colunas antes de migrações de dados
  • Comparar métodos de análise de linhagem (tabela vs coluna) em escala
  • Identificar quais ativos downstream realmente quebram ao alterar uma coluna
  • Reduzir alert fatigue em pipelines de dados críticos

Veredito

Ferramenta altamente técnica e especializada para equipes de engenharia de dados que precisam entender real impacto de mudanças. Benchmark sólido e código bem estruturado; não é para iniciantes, mas resolve um problema real e bem definido em linhagem de dados.

README

A tradução para o português deste README ainda está na fila.

data-catalog-lineage-mcp-server

typescript mcp tests scale license

A Model Context Protocol server, in TypeScript, that exposes data-catalog lineage tools over a 400,000-asset dependency graph, plus a benchmark that scores four impact-analysis methods on what actually breaks when a column changes. It speaks JSON-RPC over stdio and drops into Claude Desktop.

The challenge

"If I change this column, what breaks?" is the question every data engineer asks before a migration, and the usual answer, the full transitive set of downstream tables, is almost useless: it flags everything reachable, most of which never touches the changed column. That is alert fatigue. The precise answer needs column-level lineage, but column-level lineage metadata is usually incomplete, so it trades coverage for precision. This bench measures exactly that trade.

Use case

Use it to reason about lineage tooling: why table-level impact analysis over-alerts, how much precision column-level lineage buys, and how incomplete column metadata caps the recall you can achieve.

MCP tools

ToolDescription
search_assetsList assets in an id range
get_lineageImmediate upstream and downstream of an asset
impact_analysisAssets impacted by a column change, by chosen method
column_lineageDownstream columns derived from a given column
find_orphansNon-source assets with no downstream consumer

Headline result

400,000 assets · 3 columns each · 18% sources · 22% of column edges missing metadata · 500 change scenarios.

MethodPrecisionRecallF1Avg impactedMissed breakage
direct_dependents0.6320.7120.6122.90.288
transitive_assets0.4261.0000.52717.30.000
direct_column0.8840.5810.6571.30.419
column_lineage0.8840.7460.7902.20.254
  • Full transitive impact analysis never misses (recall 1.000) but precision is 0.43, flagging 17.3 assets when 2-3 actually break, an 8x over-alert. This is the alert-fatigue failure that makes teams ignore impact warnings.
  • Column-level lineage is the best method (F1 0.79) with 8x less noise than transitive, because it follows only the changed column's derivations.
  • But column lineage misses 25% of real breakage, and that number is no accident: it tracks the 22% of column edges whose metadata is missing. Recall is capped by metadata completeness, the single most important lever.
  • Direct (one-hop) methods are a trap: cheap, but they miss all transitive breakage (29-42% missed), which is exactly the breakage that surprises you in production.

Run it as an MCP server

npm install
npm run serve     # JSON-RPC over stdio

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "data-catalog-lineage": {
      "command": "npx",
      "args": ["tsx", "/abs/path/to/src/mcp/serve.ts"]
    }
  }
}

Smoke test:

printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | npm run -s serve

Chart gallery

Test pyramid

            /\        runner + impact orderings (runner.test)
           /  \
          /----\      MCP protocol: init/list/call/errors (mcp.test)
         /      \
        /--------\    graph generation + impact reachability (generate, impact)
       /__________\   12 tests total

Quickstart

npm test
npm run bench:fast    # 20k assets
npm run bench         # full 400k-asset run (NODE_OPTIONS=--max-old-space-size=8192)

What is in the repo

src/
  data/generate.ts    400k-asset DAG with column-level lineage (CSR typed arrays)
  models/impact.ts    ground truth + four impact methods, epoch-stamped BFS
  bench/ + runner.ts  precision/recall/F1, alert volume, missed-breakage
  mcp/server.ts       JSON-RPC MCP protocol
  mcp/tools.ts        the five catalog tools
  viz/render.ts       six SVG->PNG charts (palette "Helpdesk Sky")
  cli/main.ts         info + bench

References

  1. Anthropic (2024). Model Context Protocol Specification.
  2. Halevy, A. et al. (2016). Goods: Organizing Google's Datasets. SIGMOD.
  3. Gupta, A. et al. (2021). Data Lineage: A Survey.
  4. Armbrust, M. et al. (2020). Delta Lake. VLDB.

License

MIT.

Ver o original no GitHub

Bom para

Relacionados