convez.ai
PluginDevOps & InfraestruturaIntermediário
Vbj1808

Retrieval Lens

Vbj1808/retrieval-lens

Flight recorder local para auditoria de retrieval em agentes RAG

Ver no GitHub
InicialPluginScore · de 0 a 100

MCP server que registra e audita cada etapa de recuperação em pipelines RAG, capturando chunks, scores, fontes e rankings. Permite comparar runs de retrieval, identificar drifts de score e entender se falhas viêm da recuperação ou da geração.

Instalação

$npx convez-ai Vbj1808/retrieval-lens

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
2
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Debugar por que um agente RAG deu respostas incorretas analisando chunks recuperados
  • Comparar qualidade de retrieval antes e depois de atualizações de embeddings
  • Monitorar distribuição de scores e fontes mais relevantes ao longo do tempo

Veredito

Ferramenta bem focada para o problema real de observabilidade em RAG — entender exatamente o que o modelo viu antes de responder. Implementação limpa, local-first com SQLite, sem vendor lock-in. Ideal para quem constrói sistemas RAG produtivos.

README

A tradução para o português deste README ainda está na fila.

retrieval-lens

A black-box flight recorder for RAG retrieval inside MCP agents.

retrieval-lens is an MCP server that logs every retrieval step your RAG agent makes — what chunks were retrieved, their scores, sources, and rankings — so you can audit, replay, and diff retrieval runs after the fact.


The Problem

When a RAG agent gives a wrong answer, you need to know: did retrieval fail, or did generation fail? Right now there's no easy way to answer that. Your observability tool shows you the LLM call. It doesn't show you which chunks the model saw before it answered, what scores they had, or how retrieval changed between yesterday and today.

retrieval-lens fixes that. Every retrieval run is logged. Nothing is hidden.


Demo

When your RAG agent gives a wrong answer, ask retrieval-lens what it saw:

await mcp.call("retrieval_diff", {
  run_id_a: "support-bot-before-embedding-refresh",
  run_id_b: "support-bot-after-embedding-refresh",
  match_by: "source"
});

See docs/demo-diff.png for real output from Claude Code.


MCP Tools

ToolWhat it does
retrieval_observeLog a retrieval run — query, chunks, scores, sources, rankings
retrieval_queryReplay what the model saw before a specific answer
retrieval_diffCompare two retrieval runs — what changed, what score drifted
retrieval_statsAggregate score distributions, top sources, runs over time

Quickstart

Run retrieval-lens directly with npx:

npx retrieval-lens

Add retrieval-lens to Claude Code with one command:

claude mcp add retrieval-lens npx retrieval-lens

Then call retrieval_observe after every retrieval step in your RAG pipeline:

await mcp.call("retrieval_observe", {
  run_id: crypto.randomUUID(),
  query: "what is the refund policy?",
  chunks: [
    { content: "Refunds are processed within 5 days...", score: 0.91, source: "policy.md", rank: 1 },
    { content: "Contact support for refund requests...", score: 0.74, source: "faq.md", rank: 2 }
  ],
  pipeline_tag: "support-bot"
});

Adapters

LangChain

See examples/langchain-adapter.ts

LlamaIndex

See examples/llamaindex-adapter.ts


Why not LangSmith / Langfuse?

Those are full observability platforms. retrieval-lens is surgical:

  • Local-first — SQLite, zero signup, no data leaves your machine
  • MCP-native — one config line, works in any MCP client
  • Retrieval-only — focused on the layer where most RAG failures actually happen

Status

🚧 Active development. Harness-first build using harness engineering principles.

  • F05 — scaffold
  • F01 — retrieval_observe
  • F02 — retrieval_query
  • F03 — retrieval_diff
  • F04 — retrieval_stats

License

MIT


Ver o original no GitHub

Bom para

Relacionados