convez.ai
MCP ServerIntegração & APIsIntermediário
Nanman5

ModelPicker MCP

Nanman5/modelpicker

Fonte de verdade neutra sobre LLMs para agentes escolherem modelos com dados reais

Ver no GitHub
InicialPluginScore · de 0 a 100

Servidor MCP que fornece a agentes de IA fatos atualizados e normalizados sobre o mercado de LLMs, sem opiniões. Inclui dados sobre modelos, custos, capacidades, benchmarks e provedores com rastreamento de fontes e datas de coleta.

Instalação

$npx convez-ai Nanman5/modelpicker

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 2 dias
Versão
—
Issues abertas
0
Contribuidores
2

Casos de uso

  • Agentes de IA selecionarem modelos com base em critérios objetivos (custo, contexto, benchmarks)
  • Comparar capacidades e preços de LLMs de diferentes provedores
  • Manter decisões de modelo atualizadas com dados frescos e verificáveis

Veredito

Ferramenta essencial para construir sistemas que escolhem modelos de forma fundamentada. Remove alucinações sobre preços e contexto ao fornecer dados estruturados com rastreamento de origem — ideal para aplicações que precisam tomar decisões de modelo programaticamente sem viés de provedor.

README

A tradução para o português deste README ainda está na fila.

modelpicker

Fresh, normalized facts about LLMs — for the AI agents that have to pick one.

modelpicker is an MCP server that gives Claude, Codex, Cursor, Cline, and any other MCP-aware agent a clean source of truth about the current LLM landscape: which models exist, what they cost, what they can do, and how they score on independent benchmarks.

It is deliberately opinion-free. There is no recommend_model tool, no "winner" in compare_raw, no aggregated ranking. The agent gets raw, sourced facts and decides for itself.

Why

When you ask an agent to pick a model:

  • Its training data is months out of date.
  • It has provider bias (Claude likes Claude, GPT likes GPT).
  • It hallucinates prices, context windows, and capabilities.
  • Or it just defaults to whatever was popular a year ago.

modelpicker removes the guessing. Every fact carries the URL it came from and the date it was scraped.

Installation

Add it to your MCP-compatible client. The dataset is bundled with the package, so it works offline.

For AI agents: self-install prompt

Paste this into any agent that can edit files and run shell commands (Claude Code, Cursor, Codex, etc.):

Install the MCP server at https://github.com/Nanman5/modelpicker. Add an entry under mcpServers.modelpicker in my MCP config (Claude Code: claude mcp add modelpicker -- npx -y github:Nanman5/modelpicker; Claude Desktop: ~/Library/Application Support/Claude/claude_desktop_config.json; Cursor: .cursor/mcp.json or ~/.cursor/mcp.json; Codex: ~/.codex/mcp_servers.json). Use command: "npx" and args: ["-y", "github:Nanman5/modelpicker"]. After installing, restart the client and call the list_providers tool to verify it connected.

Manual install — Claude Code

claude mcp add modelpicker -- npx -y github:Nanman5/modelpicker

Manual install — config file (Claude Desktop / Cursor / Cline / generic .mcp.json)

{
  "mcpServers": {
    "modelpicker": {
      "command": "npx",
      "args": ["-y", "github:Nanman5/modelpicker"]
    }
  }
}

The first launch clones the repo and builds via the prepare script (~10 seconds). Subsequent launches are instant.

Smoke test (no MCP client needed)

echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}
{"jsonrpc":"2.0","method":"notifications/initialized","params":{}}
{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' | npx -y github:Nanman5/modelpicker

Should print one JSON-RPC response per request. If you see the six tools listed in the second response, you're good.

When modelpicker-mcp is published to npm, replace github:Nanman5/modelpicker with modelpicker-mcp.

What the agent gets

Six tools. Each response includes dataset_generated_at so the agent can reason about freshness.

ToolPurpose
list_modelsList models, optionally filtered by provider, capability, modality, max input price, or min context window. Excludes deprecated by default.
get_modelFull record for one model id (or alias).
list_providersAll providers in the dataset with model counts.
get_providerOne provider with all of its models embedded.
list_benchmarksRaw benchmark scores, optionally filtered by model, benchmark name, or source. No rankings, no aggregates.
compare_rawSide-by-side matrix of two or more models. No "winner".

Example get_model output:

{
  "dataset_generated_at": "2026-05-07T06:17:00.000Z",
  "model": {
    "id": "anthropic/claude-opus-4-5",
    "provider": "anthropic",
    "name": "Claude Opus 4.5",
    "aliases": ["claude-opus-latest"],
    "context_window": 1000000,
    "max_output_tokens": 64000,
    "pricing": {
      "input_per_million_usd": 15,
      "output_per_million_usd": 75,
      "cache_read_per_million_usd": 1.5,
      "cache_write_per_million_usd": 18.75
    },
    "modalities": { "input": ["text", "image"], "output": ["text"] },
    "capabilities": ["tool_use", "vision", "extended_thinking", "prompt_caching"],
    "knowledge_cutoff": "2026-01",
    "released_at": "2025-10-15",
    "deprecated": false,
    "deprecation_date": null,
    "benchmarks": [
      {
        "name": "mmlu-pro",
        "score": 84.2,
        "unit": "%",
        "source": "https://artificialanalysis.ai/models",
        "source_name": "Artificial Analysis",
        "as_of": "2026-05-07"
      }
    ],
    "last_updated": "2026-05-07",
    "sources": [
      {
        "type": "pricing_page",
        "url": "https://www.anthropic.com/pricing",
        "scraped_at": "2026-05-07T06:17:00.000Z"
      }
    ]
  }
}

V1 coverage

Providers: Anthropic, OpenAI, Google, xAI. Benchmark sources: Artificial Analysis, Aider Polyglot leaderboard.

More providers (Mistral, DeepSeek, Meta, Cohere) and more benchmark sources (LMArena, Vellum) are straightforward to add — see Contributing below.

How the dataset stays fresh

  • A GitHub Action runs daily (and on-demand) and executes the scraper pipeline.
  • Each provider has a per-file scraper in src/scrapers/. Scrapers fetch HTML, clean it, then use an LLM (NVIDIA Build API by default) to extract structured records — robust against page-layout changes.
  • A separate stage runs benchmark scrapers (src/scrapers/benchmarks/) and attaches scores to models by id / alias / name, never silently misattributing.
  • The pipeline writes data/models.json and commits it. The full history lives in git log -p data/models.json.
  • The published npm package bundles the latest committed dataset, so MCP clients work offline.

Sanitization & integrity

The page → LLM → dataset pipeline has several guards against bad data:

  • HTML cleanup — <script>, <style>, navigation chrome, hydration placeholders, inline style= and on*= attributes are stripped before text extraction. Bare URLs and zero-width / bidi characters are removed (URLs are noise; the real source URLs are tracked separately in each model's sources array).
  • Prompt-injection guard — page content is wrapped in <page url="..." type="...">…</page> and the system prompt explicitly instructs the LLM to treat anything inside those tags as data, never instructions. Literal </page> substrings in the page are escaped so a hostile page can't close the tag early.
  • Anti-hallucination — the system prompt forbids prior-knowledge fill-ins ("claude-1", "gpt-3.5"). The LLM must emit only models whose names appear verbatim in the page text.
  • Cross-provider guard — marketplace pages (Vertex AI, Bedrock, Azure) list third-party models hosted on the platform. A name-pattern check drops models whose slug clearly belongs to another provider (e.g. a claude-* model returned by the Google scraper is rejected and logged).
  • Per-row Zod validation — every extracted row is validated against the strict ModelFact schema. Invalid rows are dropped with a warning; the rest of the batch survives.
  • Pipeline tolerance — if a single page fetch, single scraper, or single benchmark source fails, the rest still run and the dataset retains prior data for the failing piece.

Together these mean a malicious or misformatted page can't poison the dataset: the worst case is "no fresh data for that provider this run, prior data kept", not "wrong data committed".

Local development

git clone <this repo>
cd modelpicker
npm install
cp .env.example .env  # add your NVIDIA API key (free at https://build.nvidia.com)

npm run typecheck
npm test

# Run the scraper pipeline locally:
npm run scrape -- --dry-run                  # see what would be written
npm run scrape -- --only anthropic           # one provider
npm run scrape -- --skip-benchmarks          # provider data only
npm run scrape -- --benchmarks-only          # refresh benchmark scores only

# Run the MCP server in dev:
npm run dev

# Build for publish:
npm run build

Environment variables

VariableRequiredDefault
NVIDIA_API_KEYyes (pipeline only)—
NVIDIA_EXTRACTION_MODELnometa/llama-3.3-70b-instruct
NVIDIA_BASE_URLnohttps://integrate.api.nvidia.com/v1
MODELPICKER_DATAnobundled data/models.json

The MCP server itself never calls the LLM — only the pipeline does. End users do not need an API key.

Contributing a new provider

  1. Add a file src/scrapers/<id>.ts that exports a Scraper constant: provider metadata, the URLs to fetch, and any extraction hints.
  2. Register it in src/scrapers/index.ts.
  3. Add a fixture-based test in tests/scrapers/ (mock the LLM and assert the run loop attaches sources correctly).

Same idea for benchmark sources under src/scrapers/benchmarks/.

Architecture

┌──────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│  Scraper pipe    │────▶│  data/models.    │────▶│   MCP server     │
│  (GH Action)     │     │  json (committed)│     │   (npm package)  │
└──────────────────┘     └──────────────────┘     └──────────────────┘
   daily / manual           single source              consumed by
                            of truth, in git           Claude/Codex/etc

Full design rationale: docs/superpowers/specs/2026-05-07-modelpicker-design.md.

License

MIT

Ver o original no GitHub

Bom para

Relacionados