convez.ai
MCP ServerDevOps & InfraestruturaIntermediário
erphq

Skillcheck

erphq/skillcheck

Linter estático para skills Claude Code — catch bugs antes do runtime

Ver no GitHub
BomPluginScore · de 0 a 100

Analisador estático para skills do Claude Code. Valida manifests YAML, verifica referências de ferramentas e servidores MCP, detecta colisões entre descrições de skills e identifica armadilhas de autoria antes do runtime. Integra-se com CI/CD para capturar ~80% dos bugs em milissegundos.

Instalação

$npx convez-ai erphq/skillcheck

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
0
Saúde
100%
Último commit
há 1 mês
Versão
—
Issues abertas
5
Contribuidores
2

Casos de uso

  • Validar manifests YAML de skills antes do deploy
  • Detectar colisões de descrição entre múltiplos skills
  • Verificar referências quebradas de ferramentas e servidores MCP
  • Falhar CI automaticamente se encontrar erros de autoria
  • Emitir relatórios estruturados (JSON) para pipelines

Veredito

Ferramenta madura e bem-pensada que fecha um gap real: feedback imediato sobre bugs em skills sem depender de testes end-to-end contra o modelo. Essencial para times que mantêm múltiplos skills em produção.

README

A tradução para o português deste README ainda está na fila.

skillcheck

static analyzer for Claude Code skills

Lint the manifest. Verify the refs. Catch the collisions before runtime.

ci tests

license status node

A linter for Claude Code skill packages. Validates the YAML frontmatter against a schema, checks every referenced tool / MCP server actually resolves, scores description-trigger collisions across a skill set, and warns about authoring pitfalls Claude won't tell you about at runtime.

Why

Skills are an authoring surface. The runtime that consumes them is opaque: an author finds out a skill is broken when an agent silently picks the wrong one, invokes a tool that no longer exists, or skips the skill entirely because three other skills described the same trigger. None of those signals come back through the model output, so the only feedback channel an author normally has is end-to-end testing against a real model, which costs a network round-trip and a billing line per check.

Static analysis closes that gap. Parse the manifest, walk the references, score the description collisions, and fail CI. Catches roughly 80% of the bugs in milliseconds for free.

✦ The bugs skillcheck catches

BugWhat happens at runtimeWhat skillcheck does
Missing descriptionSkill silently never selectedError
Tool listed in tools: was renamedSkill invokes ghost tool, failsWarn
Two skills with overlapping descriptionsWrong one firesWarn (Jaccard)
Description longer than 500 charsRegistry pollution; trigger dilutionWarn
Frontmatter name: doesn't match the fileConfused lookups, lost skillsWarn
MCP tool string typo (mcp__githhub__...)Skill invokes ghost toolError
MCP server not configuredSkill works for author, breaks for usersWarn

Every one of these is an actual bug we've seen. None show up in the model output. They show up as the skill not being selected, or being selected and then failing silently.

✦ Install

npm install -g skillcheck
# or one-shot, no install:
npx skillcheck

To run from source:

git clone https://github.com/erphq/skillcheck && cd skillcheck
npm install && npm run build
node dist/cli.js

Cutting a release: tag and create a GitHub Release named vX.Y.Z matching package.json's version. The release.yml workflow runs the full lint + build + test pipeline, then npm publish --provenance using the NPM_TOKEN repo secret.

✦ Usage

# lint every skill in the default location
skillcheck

# lint specific paths or globs
skillcheck path/to/skill.md
skillcheck "skills/**/*.md" "personal/*.md"

# fail CI on warnings
skillcheck --strict

# emit machine-readable output
skillcheck --format json

Exit codes:

CodeMeaning
0Clean (no errors; warnings allowed when --strict is off)
1One or more errors
2Warnings only, with --strict

✦ What it checks

RuleSeverityDescription
frontmatter-schemaerrorname and description are required strings; tools is array or comma-string
mcp-tool-formaterrorStrings starting with mcp__ parse as mcp__<server>__<tool>
tool-unknownwarnTool is not a known built-in and not an MCP tool
mcp-server-unknownwarnMCP tool references a server not configured in any settings.json
description-lengthwarnDescription longer than 500 chars dilutes the trigger signal
name-driftwarnFrontmatter name: doesn't match the filename or directory
description-collisionwarnTwo skills' descriptions have Jaccard ≥ 0.6
tools-overloadedwarntools: lists 10 or more entries; narrow the list to what this skill actually needs
duplicate-namewarnTwo or more skills share the same name: value; resolution is ambiguous
empty-bodywarnSkill body has no instructions; Claude has nothing to follow
model-unknownwarnmodel: value is not a recognized Claude model ID; likely a typo
parseerrorThe file doesn't have valid frontmatter / YAML

The MCP and built-in tool checks read ~/.claude/settings.json and ./.claude/settings.json to know which servers are configured. If neither exists, MCP server warnings are skipped (they'd be noise on a fresh checkout).

✦ Example output

skills/deploy.md
  warn  description-collision - description overlaps with 'release' (Jaccard 0.71)
  warn  description-length - description is 612 chars (>500)

skills/release.md
  warn  description-collision - description overlaps with 'deploy' (Jaccard 0.71)

skills/typo.md
  error frontmatter-schema - name: name is required
  warn  tool-unknown - tool 'BashShell' is not a known built-in Claude Code tool

1 error, 4 warnings, 0 info

✦ CI integration

Two-line GitHub Actions step:

- run: npx skillcheck --strict --format json > skillcheck.json
- run: cat skillcheck.json | jq '.diagnostics[] | select(.severity=="error")'

Or simpler - fail the build on any warning:

- run: npx skillcheck --strict

✦ How

flowchart LR
  files[skill files<br/>.md + frontmatter] --> P[parse<br/>YAML + body]
  P --> S[schema<br/>zod validation]
  P --> R[ref resolver<br/>tools · MCP · built-ins]
  P --> C[collision detector<br/>description tokens]
  S & R & C --> O[reporter<br/>text · json · sarif]

Pure Node, no runtime dependency on Claude Code itself. Reads ~/.claude/settings.json and .claude/settings.json to know which tools and MCP servers are configured. Designed to be vendored into a skill repo's CI.

✦ Programmatic API

import { parseSkillFile, runChecks, loadConfig } from "skillcheck";

const config = await loadConfig(process.cwd());
const skill = await parseSkillFile("skills/deploy.md");
const diagnostics = runChecks([skill], config);
console.log(diagnostics);

Useful for piping into your own reporters, custom checks, or test suites.

✦ Anti-patterns this catches (and prevents)

Anti-patternWhy it's bad
Description that's a multi-paragraph essayClaude has to read all skill descriptions every turn - long ones get skimmed
Two skills both described as "use this when working on X"Selection is non-deterministic; one of them never fires
tools: [Read, Edit, Bash, Write, ...] (everything)Defeats the purpose of tools:; signals the author didn't think about scope
name: my-skill in skills/different/different-skill.mdSkill name resolution diverges from file path
mcp__github__create_issue with no github MCP server configuredSkill works for author, breaks for everyone else

✦ Non-goals

  • Runtime evaluation. No LLM calls. We don't probe whether a skill is "good"; we only check whether it's well-formed and refer-able.
  • Skill generation. We don't write skills for you.
  • Auth or sandboxing. That's the runtime's job.
  • Plugin manifest validation. Hooks, commands, and other plugin artifacts are out of scope (a separate tool).

✦ FAQ

Q: Why are unknown tools a warning and not an error? A: The built-in tool list changes between Claude Code versions and isn't published as a stable schema. We err on the side of not breaking your CI when Anthropic ships a new tool.

Q: How do I add my own checks? A: Pass --plugin <path-or-specifier> (repeatable). A plugin is a module whose default export is { name, rules }, where each rule has an id and a check(ctx) that returns Diagnostic[]. The context exposes the same parsed / validated / config the built-in checks see, so you can write semantic checks against the real frontmatter shape. A plugin rule that throws is converted to an error diagnostic tagged <plugin>/<rule-id> so a buggy plugin cannot take down the whole run. See the SkillcheckPlugin type exported from skillcheck.

Q: What about skills inside plugin directories? A: They work - point skillcheck at the directory. Plugin-namespaced skills (plugin:skill) aren't validated against plugin manifests yet.

Q: SARIF support? A: Shipped in v0.4 (--format sarif). Output conforms to SARIF 2.1.0 and is consumable by GitHub Code Scanning's upload-sarif action. Every rule appears in tool.driver.rules with a stable id, name, short description, and default level.

Q: How does this differ from markdownlint or eslint? A: Those check syntax. skillcheck checks semantics: does this skill refer to tools that exist, does this name match its file, do two skills collide on triggers? Stylistic checks belong upstream of skillcheck.

✦ Roadmap

  • v0.0 - scaffold, schema, structure
  • v0.1 - frontmatter schema + tool ref check + tests
  • v0.2 - MCP server ref check
  • v0.3 - description collision detector (Jaccard ≥ 0.6)
  • v0.4 - SARIF 2.1.0 output for GitHub Code Scanning
  • v0.5 - npm publish + GitHub Actions release workflow (--provenance); CHANGELOG.md
  • v0.6 - --fix mode for safe auto-corrections (today: name-drift); plugin API
  • v1.0 - used by erphq/skills in CI; documented schema versioned independently

✦ Topics

claude-code · claude · ai-agents · linter · static-analysis · mcp · model-context-protocol · typescript · cli · devtools

✦ License

MIT - see LICENSE.

Ver o original no GitHub

Bom para

Relacionados