convez.ai
SkillProdutividadeAvançado
vanderheijden86

Showdown de Modelos LLM

vanderheijden86/showdown-claude-skill

Compare Claude, ChatGPT e Gemini lado a lado com julgamento cego

Ver no GitHub
InicialPluginScore · de 0 a 100

Skill para Claude Code que compara respostas de Claude, ChatGPT e Gemini para um mesmo prompt, com análise estruturada e cross-judging cego entre os modelos. Permite avaliar qual modelo performa melhor em cada contexto.

Instalação

$npx convez-ai vanderheijden86/showdown-claude-skill

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 9 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Testar qual modelo é melhor para sua tarefa específica
  • Análise comparativa de estilo e abordagem entre LLMs
  • Avaliação imparcial via cross-judging anônimo entre modelos

Veredito

Ferramenta especializada para comparação rigorosa de modelos com blind judging bem desenhado. Exige setup complexo (CLIProxyAPI, múltiplas autenticações) mas oferece insights valiosos sobre performance relativa — ideal para quem precisa escolher modelo por caso de uso.

README

A tradução para o português deste README ainda está na fila.

/showdown — Multi-Model LLM Comparison Skill

A Claude Code skill that pits Claude, ChatGPT, and Gemini against each other with the same prompt, then lets them cross-judge each other blind.

sequenceDiagram
    participant You as You
    participant CC as Claude Code
    participant AI as Claude + GPT + Gemini

    You->>CC: /showdown "your question"
    CC->>AI: Same prompt to all 3
    AI-->>CC: 3 responses
    CC-->>You: Comparison + analysis

    You->>CC: /showdown judge
    CC->>CC: Anonymize (A/B/C)
    CC->>AI: Each model judges the other 2
    AI-->>CC: Scores + verdict
    CC-->>You: Leaderboard + winner

Features

/showdown <prompt>

Sends the same prompt to all three models in parallel and presents:

  • Full responses from each model with timing
  • Structured comparison analysis (fixed template):
    • Agreement summary
    • Disagreements table (model vs model vs model)
    • Style & Approach table (tone, length, structure, examples)
    • Best Response pick with reasoning
  • Save to markdown — full responses + analysis stored in ./showdown-output/

/showdown judge

Blind cross-judging where each model evaluates the other two:

  • Anonymized — responses labeled A/B/C, models don't know who wrote what
  • Cross-judge — each model scores the other two (not its own)
  • Scored rubric — 1-10 on 5 dimensions:
    • 4 fixed: Accuracy, Depth, Clarity, Originality
    • 1 prompt-adaptive (e.g., Persuasiveness, Technical Correctness)
  • Explicit fairness instruction — judges told not to identify authorship
  • Batch reveal — anonymous verdicts shown first, then mapping revealed
  • Leaderboard — averaged scores + win count
  • Narrative synthesis — highlights agreements, disagreements, surprises
  • Appends to existing markdown — single file with showdown + judge results

Input Sources for Judge Mode

  • Same session — judges the responses from the /showdown just run
  • Load from file — lists recent files in showdown-output/ and lets you pick

Prerequisites

CLIProxyAPI

CLIProxyAPI routes requests to different LLM providers through a single OpenAI-compatible endpoint.

brew install cliproxyapi

Authenticate with each provider:

cliproxyapi --claude-login   # Claude (Anthropic)
cliproxyapi --codex-login    # ChatGPT/Codex (OpenAI)
cliproxyapi --login           # Gemini (Google)

Gemini CLI

CLIProxyAPI routes Gemini requests through the Gemini CLI, which must be installed separately:

npm install -g @google/gemini-cli

Start the proxy:

brew services start cliproxyapi
# or: cliproxyapi

Claude Code

The skill runs inside Claude Code.

Installation

Clone or copy the skill into your Claude Code skills directory:

# Option 1: Clone the repo
git clone https://github.com/vanderheijden86/showdown-claude-skill.git
ln -s "$(pwd)/showdown-claude-skill" ~/.claude/skills/showdown

# Option 2: Copy just the skill folder
git clone https://github.com/vanderheijden86/showdown-claude-skill.git
cp -r showdown-claude-skill ~/.claude/skills/showdown

Configuration

Models (scripts/models.conf)

# Format: provider_prefix|model_id|display_name
# Prefix with ! to disable a model
claude|claude-opus-4-6|Claude Opus 4.6
chatgpt|gpt-5.3-codex|GPT-5.3 Codex
gemini|gemini-3-pro-preview|Gemini 3 Pro

Swap models by editing this file. Provider prefixes determine routing:

  • claude-* routes to Claude
  • gpt-* routes to ChatGPT/Codex
  • gemini-* routes to Gemini

API Key (scripts/showdown.sh / scripts/judge.sh)

Set via environment variable or uses the default from cliproxyapi.conf:

export CLIPROXY_API_KEY="your-api-key-1"

File Structure

showdown-claude-skill/
├── README.md                 # This file
├── SKILL.md                  # Main /showdown skill definition
├── SHOWDOWN-SPEC.md          # Design spec
├── judge/
│   └── SKILL.md              # /showdown judge skill definition
├── scripts/
│   ├── models.conf           # Model configuration
│   ├── showdown.sh           # Parallel comparison script
│   └── judge.sh              # Parallel judge script
└── examples/
    └── prompting-techniques-showdown.md

Output Format

Saved markdown files in ./showdown-output/ contain:

  1. Metadata (date, models, original prompt)
  2. Full unabridged responses from each model
  3. Structured comparison analysis
  4. Judge verdicts (if judge mode was run)
  5. Leaderboard table with averaged scores
  6. Narrative synthesis

Example

# In Claude Code:
/showdown "How important are prompting techniques like prompt contracts or Socratic prompting?"

# After reviewing, run judge mode:
/showdown judge

See examples/prompting-techniques-showdown.md for a full result — three model responses, structured comparison analysis, blind cross-judging verdicts, and final leaderboard.

License

MIT

Ver o original no GitHub

Bom para

Relacionados