convez.ai
SkillDevOps & InfraestruturaAvançado
zpzjzj

Skill-up Action: Avaliação de Skills de Agentes em CI

zpzjzj/skill-up-action

Testes comportamentais multi-engine para Agent Skills em CI/CD

Ver no GitHub
InicialPluginScore · de 0 a 100

GitHub Action que executa testes comportamentais (evals) de Agent Skills em CI, validando o mesmo skill simultaneamente em múltiplos engines (Claude Code, Codex, Qoder CLI). Permite detectar regressions e melhorias em pull requests antes de merge, garantindo compatibilidade cross-engine.

Instalação

$npx convez-ai zpzjzj/skill-up-action

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Validar Agent Skills em múltiplos engines antes de merge em PR
  • Detectar regressions de comportamento entre versões de skills
  • Garantir compatibilidade cross-platform (Claude, OpenAI, Qoder) de uma mesma skill

Veredito

Ferramenta especializada e bem-arquitetada para times que desenvolvem Agent Skills em produção. Essencial quando o skill precisa rodar em múltiplos engines; integração nativa com GitHub Actions é um ponto forte, mas exige runners Linux e conhecimento técnico sólido sobre evals e CI/CD.

README

A tradução para o português deste README ainda está na fila.

Agent Skill Eval (skill-up)

Run skill-up behavioral evals on your Agent Skills in CI — and check the same skill across multiple engines (claude_code, codex, qodercli) in one action.

Skills are code for agents; they deserve the same rigor as unit tests. Edit a skill, open a PR, and this action tells you whether the change was an improvement or a regression — on every engine your users run.

Why multi-engine

Static linters (frontmatter/structure) and single-engine behavioral runners already exist. The Agent Skills format is now adopted by Claude Code, Codex, Cursor, Gemini CLI and more — so the open question for a shared skill isn't just "does it work?" but "does it work the same on codex and qoder as on claude?" This action answers that: one eval.yaml, run across engines, in PR CI.

Usage

- uses: actions/checkout@v4
- uses: zpzjzj/skill-up-action@v0.1.0
  with:
    engine: claude_code                     # or codex / qodercli; empty = let eval.yaml decide
    base-url: https://api.anthropic.com      # public endpoint; omit for the CLI default
    api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    skill-target: evals/eval.yaml

A full PR workflow (with paths filter and a cross-engine matrix) is in examples/skill-eval.yml.

Inputs

inputdefaultnotes
engine''claude_code / codex / qodercli; empty -> eval.yaml declares it
model''passed to --model when set
provider''resolves base-url from a built-in table (public: dashscope)
api-key''routed per engine, or passed via --api-key when engine empty. Use a secret.
base-url''public endpoint; defaults to the agent CLI's own
open-sandbox-api-key''only for the opensandbox eval environment
skill-target.skill dir or eval config path
skill-up-versionlatestignored when the image already ships skill-up
skill-up-commandskill-up runmust start with skill-up
parallelism''--parallelism (1-256)
agent-install-command''for custom images missing an engine CLI

Outputs

outputnotes
exit-codeskill-up exit code (0 = all passed)
report-dirdir with JSON / JUnit / HTML reports

Credentials

Pass api-key from a GitHub secret. The action routes it per engine:

engineapi-key ->base-url ->
claude_codeANTHROPIC_API_KEY (+ ANTHROPIC_AUTH_TOKEN)ANTHROPIC_BASE_URL
codexOPENAI_API_KEYOPENAI_BASE_URL
qodercliQODER_PERSONAL_ACCESS_TOKEN(none)

Notes

  • Linux runners only — this is a Docker container action.
  • The runner image (ghcr.io/<owner>/skill-up-runner) bakes claude / codex / qodercli / skill-up; it is built and pushed by .github/workflows/build-image.yml.

License

Apache-2.0.

Ver o original no GitHub

Bom para

Relacionados