Skill-up Action: Avaliação de Skills de Agentes em CI
zpzjzj/skill-up-action
Testes comportamentais multi-engine para Agent Skills em CI/CD
Ver no GitHubGitHub Action que executa testes comportamentais (evals) de Agent Skills em CI, validando o mesmo skill simultaneamente em múltiplos engines (Claude Code, Codex, Qoder CLI). Permite detectar regressions e melhorias em pull requests antes de merge, garantindo compatibilidade cross-engine.
Instalação
npx convez-ai zpzjzj/skill-up-action1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Validar Agent Skills em múltiplos engines antes de merge em PR
- Detectar regressions de comportamento entre versões de skills
- Garantir compatibilidade cross-platform (Claude, OpenAI, Qoder) de uma mesma skill
Veredito

Ferramenta especializada e bem-arquitetada para times que desenvolvem Agent Skills em produção. Essencial quando o skill precisa rodar em múltiplos engines; integração nativa com GitHub Actions é um ponto forte, mas exige runners Linux e conhecimento técnico sólido sobre evals e CI/CD.
README
A tradução para o português deste README ainda está na fila.
Agent Skill Eval (skill-up)
Run skill-up behavioral evals on your
Agent Skills in CI — and check the same
skill across multiple engines (claude_code, codex, qodercli) in one
action.
Skills are code for agents; they deserve the same rigor as unit tests. Edit a skill, open a PR, and this action tells you whether the change was an improvement or a regression — on every engine your users run.
Why multi-engine
Static linters (frontmatter/structure) and single-engine behavioral runners
already exist. The Agent Skills format is now adopted by Claude Code, Codex,
Cursor, Gemini CLI and more — so the open question for a shared skill isn't just
"does it work?" but "does it work the same on codex and qoder as on claude?"
This action answers that: one eval.yaml, run across engines, in PR CI.
Usage
- uses: actions/checkout@v4
- uses: zpzjzj/skill-up-action@v0.1.0
with:
engine: claude_code # or codex / qodercli; empty = let eval.yaml decide
base-url: https://api.anthropic.com # public endpoint; omit for the CLI default
api-key: ${{ secrets.ANTHROPIC_API_KEY }}
skill-target: evals/eval.yaml
A full PR workflow (with paths filter and a cross-engine matrix) is in
examples/skill-eval.yml.
Inputs
| input | default | notes |
|---|---|---|
engine | '' | claude_code / codex / qodercli; empty -> eval.yaml declares it |
model | '' | passed to --model when set |
provider | '' | resolves base-url from a built-in table (public: dashscope) |
api-key | '' | routed per engine, or passed via --api-key when engine empty. Use a secret. |
base-url | '' | public endpoint; defaults to the agent CLI's own |
open-sandbox-api-key | '' | only for the opensandbox eval environment |
skill-target | . | skill dir or eval config path |
skill-up-version | latest | ignored when the image already ships skill-up |
skill-up-command | skill-up run | must start with skill-up |
parallelism | '' | --parallelism (1-256) |
agent-install-command | '' | for custom images missing an engine CLI |
Outputs
| output | notes |
|---|---|
exit-code | skill-up exit code (0 = all passed) |
report-dir | dir with JSON / JUnit / HTML reports |
Credentials
Pass api-key from a GitHub secret. The action routes it per engine:
| engine | api-key -> | base-url -> |
|---|---|---|
claude_code | ANTHROPIC_API_KEY (+ ANTHROPIC_AUTH_TOKEN) | ANTHROPIC_BASE_URL |
codex | OPENAI_API_KEY | OPENAI_BASE_URL |
qodercli | QODER_PERSONAL_ACCESS_TOKEN | (none) |
Notes
- Linux runners only — this is a Docker container action.
- The runner image (
ghcr.io/<owner>/skill-up-runner) bakes claude / codex / qodercli / skill-up; it is built and pushed by.github/workflows/build-image.yml.
License
Apache-2.0.