convez.ai
SkillDesenvolvimento WebIntermediário
Arize-ai

Arize Skills

Arize-ai/arize-skills

Skills para observabilidade e debug de LLM apps via CLI ax

Ver no GitHub
BomPluginScore · de 0 a 100

Skills para agentes de IA que adicionam observabilidade, rastreamento e otimização de prompts em aplicações LLM. Encapsula workflows refinados da plataforma Arize, facilitando instrumentação com traces, execução de experimentos e debug de apps em produção sem lidar manualmente com quirks da API.

Instalação

$npx convez-ai Arize-ai/arize-skills

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
74
Forks
14
Acompanhando
1
Saúde
37%
Último commit
há 2 dias
Versão
v1.4.0
Issues abertas
38
Contribuidores
17

Casos de uso

  • Adicionar rastreamento distribuído em aplicações LLM
  • Debugar latência e erros em produção com análise de traces
  • Executar experimentos A/B em prompts e modelos

Veredito

Excelente para times que usam LLMs em produção e precisam visibilidade operacional. Integração robusta com múltiplos agentes de código (Cursor, Claude Code, Copilot), mas requer CLI ax e credenciais Arize — setup inicial é moderado.

README

A tradução para o português deste README ainda está na fila.

Arize Skills

Skills that guide AI coding agents to help you add observability, run experiments, and optimize prompts for your LLM applications.

These skills encode the workflows we've refined building the Arize platform and helping teams debug LLM apps in production. They handle the ax CLI flags, data shape quirks, and multi-step recipes so you don't have to.

Works with Cursor, Claude Code, Codex, GitHub Copilot, Windsurf, and 40+ other agents.

New to Arize? Start Here

Adding tracing to your app — give your coding agent this prompt:

Follow the instructions from https://arize.com/docs/PROMPT.md and ask me questions as needed.

This walks through a two-phase flow: analyze your codebase for LLM providers and frameworks, then add Arize AX tracing with the right instrumentors. No skill installation needed.

Already have traces? Give your agent this prompt to install the skills and start debugging:

Install the Arize skills plugin from https://github.com/Arize-ai/arize-skills, then use the arize-trace skill to export and analyze recent traces from my project. Summarize any errors or latency issues you find.

Installation

Option 1: npx (recommended)

# Interactive — choose skills, agent, and scope
npx skills add Arize-ai/arize-skills

# Non-interactive — install everything with auto-detected defaults
npx skills add Arize-ai/arize-skills --skill "*" --yes

Both options auto-detect your agent (Cursor, Claude Code, Codex, etc.) and symlink skills into place.

Option 2: git clone

macOS / Linux:

git clone https://github.com/Arize-ai/arize-skills.git
cd arize-skills
./install.sh --project ~/my-project

Windows (PowerShell):

git clone https://github.com/Arize-ai/arize-skills.git
cd arize-skills
.\install.ps1 -Project ~\my-project

The installer detects installed agents and optionally installs the ax CLI. Use --global / -Global instead to install to ~/.<agent>/skills/.

Option 3: ax CLI

If you already have ax installed (v0.9.0+):

ax skills install

Option 4: Claude Code plugin

Open up a Claude Code session and execute the following:

/plugin marketplace add Arize-ai/arize-skills
/plugin install arize-skills@arize-skills

Prerequisites

Arize CLI (ax)

The skills use the ax CLI to interact with the Arize API. Install it if you don't have it:

# Preferred (isolated environment)
uv tool install arize-ax-cli
# or
pipx install arize-ax-cli
# Fallback
pip install arize-ax-cli

Authentication

Option A — ax CLI profile (recommended):

Set up your API key once and it persists across all sessions and projects:

# Interactive wizard (creates 'default' profile if no profiles exist)
ax profiles create

# Or pass the key directly (optional profile name as positional arg)
ax profiles create --api-key YOUR_API_KEY
ax profiles create staging --api-key YOUR_STAGING_KEY

# Update an existing profile (patches only what you specify)
ax profiles update --api-key NEW_API_KEY
ax profiles update --region us-east-1b

# Other profile management
ax profiles list
ax profiles show
ax profiles use staging
ax profiles delete staging

You'll also need a space name or ID. Find yours by running ax spaces list -o json (use the name or base64 id), then persist it:

# macOS/Linux — add to ~/.zshrc or ~/.bashrc
export ARIZE_SPACE="my-workspace"        # name, or base64 ID like U3BhY2U6...

Option B — Environment variables:

export ARIZE_API_KEY="your-api-key"       # from https://app.arize.com/admin > API Keys
export ARIZE_SPACE="my-workspace"         # space name or base64 ID from ax spaces list
# export ARIZE_DEFAULT_PROJECT=my-project # optional default project
# export OPENAI_API_KEY="sk-..."          # for AI integrations and evaluators
# export ANTHROPIC_API_KEY="sk-ant-..."   # for AI integrations and evaluators

Verify

ax --version && ax profiles show 2>&1

On-prem / self-hosted Arize

Point the CLI at your deployment with a single-endpoint profile (hostname and HTTPS port, usually 443). Replace the host with the value your operations team provides:

ax profiles create my-onprem --api-key <key> --single-host arize.yourcompany.com --single-port 443
ax profiles use my-onprem
ax profiles validate

For interactive setup, ax profiles create also offers Advanced → Single endpoint. More options (TOML, Flight/OTLP splits) are documented in the arize-ax-cli README.

Available Skills

SkillDescription
arize-traceExport traces and spans by trace ID, span ID, or session ID. Debug LLM application issues.
arize-instrumentationAdd Arize AX tracing to an app. Two-phase flow: analyze codebase, then implement instrumentation (uses Agent-Assisted Tracing).
arize-datasetCreate, manage, and download datasets and examples.
arize-experimentRun and analyze experiments against datasets.
arize-evaluatorCreate LLM-as-judge evaluators, run evaluation tasks, and set up continuous monitoring.
arize-ai-provider-integrationCreate and manage LLM provider credentials (OpenAI, Anthropic, Azure, Bedrock, Vertex, and more).
arize-annotationCreate and manage annotation configs (categorical, continuous, freeform); bulk-annotate project spans via the Python SDK.
arize-prompt-optimizationOptimize prompts using trace data, experiments, and meta-prompting.
arize-promptsManage Prompt Hub templates and versions with ax prompts (create, versions, labels).
arize-linkGenerate deep links to traces, spans, and sessions in the Arize UI.
arize-compliance-auditAudit an AI agent for regulatory compliance (EU AI Act, NIST AI RMF, GDPR, HIPAA). Produces a tailored remediation checklist.
arize-adminManage organizations, spaces, roles, role bindings, resource restrictions, and API keys. Enterprise admin workflows: team onboarding, SAML/SSO role mapping, service key management, and access control.

[!WARNING] arize-compliance-audit is for guidance only and does not constitute legal advice or a complete compliance assessment. It identifies common technical patterns based on publicly available regulatory frameworks and cannot account for your organisation's specific legal obligations, contractual commitments, or operational context. Always consult a qualified attorney or compliance specialist for binding assessments.

Installer Flags

Bash (install.sh):

FlagDescription
--project <dir>Required. Target project directory for skill symlinks
--globalInstall to ~/.<agent>/skills/ instead (alternative to --project)
--copyCopy files instead of symlinking
--forceOverwrite existing skills
--skip-cliDon't install ax CLI even if missing
--agent <name>Manually specify agent (cursor, claude, codex, copilot) — repeatable
--skill <name>Only install/uninstall specific skills — repeatable (e.g. --skill arize-trace --skill arize-dataset)
--yesSkip confirmation prompts
--listList all available skills and exit
--uninstallRemove previously installed skill symlinks

PowerShell (install.ps1):

FlagDescription
-Project <dir>Required. Target project directory for skill symlinks
-GlobalInstall to ~/.<agent>/skills/ instead (alternative to -Project)
-CopyCopy files instead of symlinking
-ForceOverwrite existing skills
-SkipCliDon't install ax CLI even if missing
-Agent <name>Manually specify agent (cursor, claude, codex, copilot) — repeatable
-Skill <name>Only install/uninstall specific skills — repeatable
-YesSkip confirmation prompts
-UninstallRemove previously installed skill symlinks
-ListList all available skills and exit

Updating

  • npx path: npx skills update
  • git clone path: cd arize-skills && git pull (symlinks update automatically)

Testing Skills

tests/run_skill.py is an interactive test harness that runs a skill end-to-end using the Claude Agent SDK. It creates a temporary workspace, passes in your Arize credentials, and streams the agent's output.

Export the required environment variables before running:

export ARIZE_API_KEY="your-api-key"       # from https://app.arize.com/admin > API Keys
export ARIZE_SPACE="my-workspace"         # space name or base64 ID from ax spaces list
export TEST_PROJECT_NAME="my-project"     # optional: Arize project for trace tests
export TEST_MODEL="claude-sonnet-4-6"     # optional: Claude model override

Then run the harness:

python tests/run_skill.py --skill arize-trace --prompt "Export trace abc123"

[!WARNING] Configure .claude/settings.json before running the test harness

The test harness uses Claude Code's bypassPermissions mode, which skips all interactive approval prompts. This is safe because the agent runs in a sandboxed temporary workspace — but only if your settings.json has a denylist blocking dangerous shell commands.

Without this, bypassPermissions gives the agent unrestricted shell access.

Add the following to .claude/settings.json in this repo (create it if it doesn't exist):

{
  "permissions": {
    "deny": [
      "Bash(rm -rf*)",
      "Bash(curl*)",
      "Bash(wget*)",
      "Bash(ssh*)",
      "Bash(scp*)",
      "Bash(git push*)",
      "Bash(sudo*)",
      "Bash(chmod*)",
      "Bash(chown*)"
    ]
  }
}

Links

License

MIT

Ver o original no GitHub

Bom para

Relacionados