convez.ai
MCP ServerIntegração & APIsAvançado
Kaydenletk

MCP Docs Assistant

Kaydenletk/mcp-docs-assistant

RAG agentico versionado para o MCP SDK TypeScript com refusal garantido

Ver no GitHub
InicialPluginScore · de 0 a 100

Assistente RAG agentico especializado no SDK TypeScript do Model Context Protocol que fornece respostas precisas e citadas, com suporte explícito às versões v1 e v2 da API. Recusa responder quando a documentação não cobre o tópico, em vez de alucinar, usando um retrieval pipeline híbrido (busca vetorial + lexical) com reranking por LLM.

Instalação

$npx convez-ai Kaydenletk/mcp-docs-assistant

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
2
Acompanhando
0
Saúde
85%
Último commit
há 4 meses
Versão
—
Issues abertas
7
Contribuidores
3

Casos de uso

  • Consultar documentação do SDK MCP com precisão de versão (v1/v2)
  • Obter respostas citadas diretamente do código-fonte
  • Evitar respostas alucinárias em tópicos não documentados

Veredito

Solução técnica robusta que resolve um problema real: assistentes genéricos que misturam versões de API incompatíveis. O design de 'refusal como código' (retrieval vazio força rejeição) é uma abordagem pragmática; eval com 18 casos auditados fornece confiança legítima. Essencial para quem trabalha com MCP SDK, limitado para usuários fora desse ecossistema.

README

A tradução para o português deste README ainda está na fila.

MCP SDK Docs Assistant

An agentic RAG assistant for the Model Context Protocol TypeScript SDK that answers "how do I…?" with version-correct (v1 vs v2), cited answers — and refuses when the docs don't cover it.

▶ Live demo: library-assisstant-ai.vercel.app

Hero


Why this exists

The MCP TypeScript SDK recently went v1 → v2 with breaking API changes (server.tool() → registerTool(), SSE → Streamable HTTP, one package → three). Generic doc bots (Context7, DeepWiki) have two failure modes that bite developers:

  1. Version-blind — they mix v1 and v2 snippets, so copy-pasted code breaks.
  2. They never refuse — ask anything and they hallucinate a confident answer.

This assistant fixes both. That's the differentiator:

DisciplineHow it's enforced
Version-correctevery chunk is tagged v1/v2; retrieval filters by version; answers label and split them
Citedevery claim links the exact GitHub source line
Refusesenforced in code, not the prompt — see below

Answer with version-coded citations


The idea worth stealing: refusal as code, not a prompt

Telling a model "don't hallucinate" is a suggestion it ignores under pressure. Instead, the retrieval tool returns empty when nothing clears a confidence bar — so the model has no material to fabricate from and is forced to refuse:

const candidates = await hybridSearch(query, { version, limit: 12 });
if (!hasConfidentMatch(candidates)) {          // best cosine sim < 0.45
  return { relevant: false, results: [] };     // model must refuse
}
const results = await rerank(query, candidates, 6);
return { relevant: true, results };

Architecture

                       ┌──────────── retrieval pipeline ────────────┐
  question ──► agent ──►│ hybrid search → refusal gate → LLM rerank  │──► cited answer
  (Gemini)    (tool     │ vector + lexical   ↑ refuse    flash-lite  │     or refusal
              calling)  │ fused by RRF       here        scores 0–10 │
                       └─────────────────────────────────────────────┘
                                         │
                                  Postgres + pgvector
                                  570 chunks: 58 v1 · 158 v2 · 354 spec
  • Ingestion (offline): clone the SDK repos (v1 + v2) and the protocol spec → chunk by heading → embed → store, each chunk tagged with version + source.
  • Retrieval: semantic (vector cosine) fused with lexical (Postgres full-text) via Reciprocal Rank Fusion, then an LLM cross-encoder rerank.
  • Generation: an agent (generateText + a searchDocs tool, bounded by stepCountIs) that cites, splits versions, and refuses.

Three surfaces from one codebase: web chat, CLI, and an MCP server.


Proven, not just demoed

An 18-case golden set (eval/) scores the behaviours that matter. pnpm eval runs every case through the live agent and writes a scorecard to docs/eval/REPORT.md:

Overall pass        18/18 (100%)
Refusal accuracy     5/5 (100%)
Answer + citation   13/13 (100%)
Version-correctness  5/5 (100%)

100% is against this repo's own curated golden set — it proves the system holds its three-discipline contract on representative questions, not that it's omniscient. Expanding the set is how regressions get caught.

The harness earned its keep: an earlier run scored 83% — two answerable questions were false-refused despite strong retrieval (top hits 0.70–0.72, well above the 0.45 gate). The model was over-refusing by re-judging relevance itself; the fix was to make refusal defer to the tool's signal. Re-run: 100%, refusal accuracy still 100%.

How it stacks up against Context7 / DeepWiki on version-isolation and refusal — the axes generic doc bots can't hold — is in docs/eval/BENCHMARK.md.


Meta: it's also an MCP server

The assistant publishes itself as an MCP server, so Claude Desktop / Cursor can call it as a tool — an MCP tool that teaches the MCP SDK.

pnpm mcp   # stdio server: ask_mcp_docs + search_mcp_docs

Tech stack

  • Next.js 16 (App Router) · AI SDK v6 (generateText/streamText, tool calling, embed)
  • Google Gemini — gemini-2.5-flash (chat), flash-lite (rerank), gemini-embedding-001 (1536-d)
  • Neon Postgres + pgvector (HNSW cosine) · Drizzle ORM
  • Zod · Vitest (58 unit tests) · @modelcontextprotocol/sdk
  • LangGraph — optional Corrective-RAG variant (see below)

Bonus: a LangGraph Corrective-RAG variant

The product agent is a lean AI SDK tool-loop. As a showcase of where a graph framework actually earns its place, lib/agent/graph.ts reimplements the agent as an explicit LangGraph state machine that grades its own retrieval and re-queries when it's weak:

rewrite → retrieve → grade ─┬─ confident ──────────► generate ─► END
   ▲                        ├─ weak, tries left ────► (loop back)
   └────────────────────────┘
                            └─ weak, out of tries ──► refuse  ─► END

It reuses the same hybrid retrieval + rerank + refusal gate, so the moat is identical — the graph only adds the self-correcting loop. Run: pnpm answer:graph "…".


Run it

pnpm install
cp .env.example .env            # add DATABASE_URL + GOOGLE_GENERATIVE_AI_API_KEY
pnpm db:setup && pnpm db:push   # pgvector + schema + full-text index
pnpm ingest                     # clone + embed the corpora (one-time)

pnpm dev                        # web chat at localhost:3000
pnpm answer "how do I register a tool?"   # CLI (AI SDK agent)
pnpm answer:graph "how do I register a tool?"  # CLI (LangGraph Corrective-RAG)
pnpm eval                       # scorecard
pnpm mcp                        # MCP server

Deploy + observability: see DEPLOY.md. Engineering log: STATUS.md.

Roadmap / help wanted

Contributions welcome — see CONTRIBUTING.md and the good first issue label.

  • LLM-as-judge faithfulness scoring in the eval harness (RAGAS-style)
  • Benchmark methodology vs Context7 / DeepWiki — docs/eval/BENCHMARK.md (live comparison columns: help wanted)
  • More corpora (additional protocol revisions, other SDK languages)
  • Conversation history persistence
  • Dark theme
  • Show retrieval/tool steps live in the UI

Contributing

See CONTRIBUTING.md and the Code of Conduct.

License

MIT © Khanh Le

Ver o original no GitHub

Bom para

Relacionados