Rubric Evaluator
halfmoon-mind/rubric-evaluator
Avalia skills com rubric determinístico e relatórios corrigíveis
Ver no GitHubSkill para avaliar estruturalmente skills do Claude Code e Codex contra um rubric determinístico de 6 seções e 30 itens. Executa verificações estruturais e de segurança com Python stdlib, aplica checks semânticos bundled e gera relatórios com grades S/A/B/C/F e recomendações de correção.
Instalação
npx convez-ai halfmoon-mind/rubric-evaluator1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Validar qualidade estrutural de diretórios de skills antes de publicação
- Detectar bloqueadores de segurança em skills (vazamento de secrets, etc)
- Auditoria automática de conformidade contra padrão de qualidade
- Gerar relatórios com ações específicas para melhorar skills
Veredito

Ferramenta prática para padronização e qualidade de skills, com verificações determinísticas stdlib-only e rubric baseado em boas práticas da indústria. Essencial para equipes mantendo diretórios de skills em Claude Code e Codex.
README
A tradução para o português deste README ainda está na fila.
Rubric Evaluator

Grade a skill directory (one that contains a SKILL.md) against a 6-section,
30-item rubric. Deterministic structural and safety checks run as a stdlib-only
Python script; semantic checks are applied from a bundled rubric reference. The
headline grade (S/A/B/C/F) is computed deterministically, never eyeballed.
The rubric is based on Toss Tech's article Skill 품질 관리를 위한 Rubric 설계와 시스템 구현.
The same skill bundle ships for two hosts from one source of truth at
plugins/rubric-evaluator/:
- Claude Code —
.claude-plugin/plugin.json+ the marketplace at the repo root - Codex —
.codex-plugin/plugin.json+.agents/plugins/marketplace.json
Install (Claude Code)
/plugin marketplace add halfmoon-mind/rubric-evaluator
/plugin install rubric-evaluator@rubric-evaluator
Then ask Claude to evaluate a skill, e.g. "grade the skill directory at ./my-skill".
To try it without installing (single session, from a local checkout):
claude --plugin-dir ./plugins/rubric-evaluator
Install (Codex)
Add this repository as a Codex marketplace source, then install the plugin:
codex plugin marketplace add halfmoon-mind/rubric-evaluator
codex plugin add rubric-evaluator@rubric-evaluator
To install from a local checkout instead, add the repo root as the local marketplace source:
codex plugin marketplace add .
codex plugin add rubric-evaluator@rubric-evaluator
Start a new Codex thread after installing, then ask Codex to evaluate a skill, e.g. "Use $rubric-evaluator to grade the skill directory at ./my-skill".
Usage
Point the skill at any directory that contains a SKILL.md. There is nothing
to run by hand — describe what you want in natural language and the skill
triggers on requests to grade, audit, review, dogfood, or improve a skill.
Example prompts:
- "Grade the skill directory at ./my-skill"
- "Audit the SKILL.md in ./plugins/foo/skills/foo and tell me what to fix"
- "Review ./my-skill against the rubric"
On Codex, prefix with the plugin handle, e.g. "Use $rubric-evaluator to grade ./my-skill".
To try it against a bundled example, point it at one of the test fixtures:
- "Grade tests/rubric-evaluator/fixtures/clean" — a well-formed skill, grades S
- "Grade tests/rubric-evaluator/fixtures/secret-leak" — trips a safety
BLOCKER, grades F
Behind the scenes the skill runs the deterministic checks, applies the semantic model checks, computes the grade, and returns a report (see What you get).
What you get
A grade plus a per-item report. For every failing item it states why it matters and how to fix it:
BLOCKER≥ 1 → F- no
BLOCKER, 0MAJOR→ S · 1–2 → A · 3–4 → B · 5+ → C MINORitems are advisory and never change the grade
Develop / test
The checker is stdlib-only (no install step). Run the test suite with:
python3 -m unittest discover -s tests/rubric-evaluator -p "test_*.py"