claude-md-directory
alex-jb/claude-md-directory
Brier-audited directory of 1000+ CLAUDE.md and skills/*.md files. Third-party honest 'does this skill actually help?' scoring.
Instalação
npx convez-ai alex-jb/claude-md-directory1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
README
A tradução para o português deste README ainda está na fila.
claude-md.directory
English | 中文 · Live: alex-jb.github.io/claude-md-directory
Brier-audited directory of 1000+
CLAUDE.mdandskills/*.mdfiles — every entry comes with a "does this skill actually help?" score, scored independently against a standardized eval harness.
First audit results (2026-06-10)
26 entries audited: 8 helpful · 15 neutral · 3 harmful against no-skill baseline on the frozen v1.0 eval set.
| Verdict | Count | Examples |
|---|---|---|
| helpful | 8 | Sahil /marketing-plan · Sahil /minimalist-review · Sahil /validate-idea (all Brier 0.05 vs 0.25) · Anthropic /frontend-design (0.30 vs 0.45) · Anthropic /internal-comms (0.30 vs 0.45) |
| neutral | 15 | Karpathy CLAUDE.md · Sahil /pricing · 11 other Anthropic skills (canvas-design, claude-api, docx, pdf, pptx, etc.) |
| harmful | 3 ⚠️ | Sahil /grow-sustainably (0.35 vs 0.25) · Anthropic /algorithmic-art (0.35 vs 0.25) · Anthropic /doc-coauthoring (0.40 vs 0.25) |
Yes, one of Sahil's 10 skills and two of Anthropic's 15 audit as harmful against no-skill baseline on our frozen 5-task eval set. The author counter-eval slot is open per docs/brier-method.md. These results almost certainly reflect domain mismatch — algorithmic-art is not built for code-review tasks. But that's the point: "this skill is great for what it's built for" is exactly what authors say without proof, and exactly what a directory can audit. The v1.1 eval set will widen to 40 tasks across all 8 domains so cross-domain skills can be scored on their home turf.
This is the kind of result a third-party directory exists to surface.
Sahil Lavingia's slavingia/skills repo hit 4.8K stars in days in March 2026 because nobody knew which skills to actually install. Anthropic Skills Marketplace launched May 2026 with 600+ free skills and no eval layer. Karpathy's CLAUDE.md got 109K stars from a single tweet — but that's N = 1.
The market has skills. It does not have an honest "does this skill actually help?" answer. This directory is that answer.
What this is
A curated, third-party, Brier-audited directory of real-world CLAUDE.md files and Claude Code skills.
Every entry includes:
- Source URL (GitHub repo + path, exact commit hash)
- One-line skill purpose ("makes Claude refuse to write
--no-verify") - Standardized eval verdict:
helpful/neutral/harmfulon a fixed eval set of 40 representative tasks - Win-rate vs no-skill baseline (Brier score, lower = better calibrated, < 0.25 = better than coin flip)
- First-seen date + last-updated date
- One-line author note (optional, fork-only)
We do not score the author. We score the skill, on a frozen eval set, with the model and version pinned. The skill author can submit a counter-eval, and we publish it next to ours.
Try it (quick demo)
git clone https://github.com/alex-jb/claude-md-directory
cd claude-md-directory
# See all 100+ seeded skills:
cat entries/*.yaml | grep -E '^name:|^win_rate:' | head -40
# Submit a new skill (one yaml file per entry):
cp entries/_template.yaml entries/my-new-skill.yaml
# ... edit + open PR
# Run our standardized eval against your skill (requires Anthropic key):
python scripts/audit.py entries/my-new-skill.yaml
Why "directory" not "marketplace"
- Marketplaces (Anthropic, GPT Store) sell their own products. They have a conflict of interest re: ranking.
- Awesome-lists are taste-driven. No grading, just curation.
- Directories are the third option: third-party, standardized scoring, public method, public dispute resolution.
Yelp for skills, not the App Store.
Method (Brier audit)
See docs/brier-method.md for the full method. Short version:
- We froze a 40-task eval set covering 8 common Claude Code domains (write code / refactor / debug / review / spec / ship / explain / triage).
- Each task has a known-good outcome scored 0..1 (some are binary; some are multi-aspect).
- We run each skill against the eval set 5 times with
claude-haiku-4-5(cheap baseline) andclaude-sonnet-4-6(strong baseline). Win-rate is averaged. - Brier score = average of
(p - actual)^2across all 40 tasks. Lower is better. < 0.25 = better than coin flip. - We publish every score, every prompt, every model output. Reproducible from a single command.
The eval set is frozen at v1.0 to prevent gaming. We'll cut v1.1 only when v1.0 scores are saturated.
Cross-stack
- council-diff — 5-voice debate library writes the per-skill explanation when there's disagreement
- solo-founder-os —
sfos-evalcron runs the periodic audit - whocalleditright — sister project, same Brier discipline applied to hedge fund manager forecasts
Status
- Repo skeleton + entry yaml schema
- 10-entry seed (Karpathy + Sahil + 8 misc)
- 100-entry seed (this week)
-
scripts/audit.pyBrier eval harness (next week) - Static site at claude-md.directory (week 2)
- Anthropic Skills Marketplace import (week 3)
- 1000-entry milestone (month 1)
Submitting a skill
- Fork the repo
- Copy
entries/_template.yamltoentries/<your-skill-slug>.yaml - Fill in
source_url,purpose,author(optional) - Open a PR. We run our standardized audit on merge and post the verdict back to your repo as a comment.
We accept counter-evals if you disagree with our score. We publish both.
License
MIT. Entries inherit their source repo's license.
Author
Built by Alex Ji (@alex-jb) — solo founder, MS CS Yeshiva, builder of council-diff + Solo Founder OS + whocalleditright. Calibration-honest by default.