convez.ai
PluginDevOps & InfraestruturaIntermediário
Rul1an

llm-cost

Rul1an/llm-cost

Análise estática de custos para LLMs, como Infracost para IA

Ver no GitHub
InicialPluginScore · de 0 a 100

Ferramenta de análise estática de custos para workloads de LLM. Estimula gastos, aplica orçamentos e compara custos em CI/CD — funciona offline e é distribuída como um único binário em Zig.

Instalação

$npx convez-ai Rul1an/llm-cost

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
1
Saúde
100%
Último commit
há 10 meses
Versão
v1.9.0
Issues abertas
75
Contribuidores
1

Casos de uso

  • Estimar custos de prompts antes da produção
  • Enforçar orçamentos em pipelines CI/CD
  • Detectar desvios entre custos estimados e faturamento real
  • Comparar impacto de cache em diferentes cenários

Veredito

Ferramenta sólida e madura para governança de custos de LLM em produção. Offline-first e zero-dependency torna a adoção simples; integração nativa com GitHub Actions e suporte a SARIF elevam o padrão para times que já usam IA em escala.

README

A tradução para o português deste README ainda está na fila.

llm-cost

Static cost analysis for LLM workloads. Estimate spend, enforce budgets, diff costs in CI/CD.

CI Release License Website

30 Seconds to Value

# No config needed
llm-cost estimate prompt.txt --model gpt-4o
prompt.txt
  Tokens: 1,847 input + 523 output (est.)
  Cost:   $0.0041 (gpt-4o)

See Cache Impact

llm-cost estimate prompt.txt --model gpt-4o --scenario cached --cache-hit-ratio 0.6
Scenario    Cost      Savings
default     $0.0052   —
cached@60%  $0.0033   -37%

Installation

Linux & macOS

curl -sSfL https://get.llm-cost.dev | sh

Windows

Windows is not currently supported via the install script.

Options:

  1. WSL2 (recommended): Use the Linux install method.
  2. Manual: Download llm-cost-windows-x86_64.exe from Releases.

CI/CD on Windows:

- name: Install llm-cost
  run: |
    Invoke-WebRequest -Uri "https://github.com/rul1an/llm-cost/releases/latest/download/llm-cost-windows-x86_64.exe" -OutFile "llm-cost.exe"
    .\llm-cost.exe --version

CI/CD Integration

llm-cost provides a native GitHub Action for seamless integration.

1. CI-Native (SARIF Reporting)

Fails the build on policy violations and uploads results to GitHub Advanced Security.

permissions:
  contents: read
  security-events: write

steps:
  - uses: actions/checkout@v4

  - name: llm-cost check (SARIF)
    uses: Rul1an/llm-cost@v1
    with:
      command: check
      format: sarif
      # Optional: path to actuals
      # actuals: usage.csv
      # Fail build on error-level violations
      fail-on-error: true

  - name: Upload SARIF
    if: always()
    uses: github/codeql-action/upload-sarif@v3
    with:
      sarif_file: llm-cost.sarif
      category: llm-cost

2. Pull Request Commenter (Legacy)

Performs budget checks and posts a sticky comment on the PR.

- uses: Rul1an/llm-cost@v1
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    # Optional: absolute budget enforcement
    budget: "10.00"
    # Optional: fail if cost increases
    fail-on-increase: true

Project Setup

llm-cost init                      # Generate manifest from prompt files
llm-cost check --budget 5.00       # Local budget enforcement
llm-cost diff --base main          # Compare costs vs branch

Example llm-cost.toml:

[defaults]
model = "gpt-4o"

[[prompts]]
path = "prompts/search.txt"
prompt_id = "search"
tags = { team = "platform", app = "search" }

Calibration (Drift Analysis)

Close the loop by comparing your estimates against actual billing data (FOCUS v1.0 CSV). Detect "Shadow AI" (unapproved usage) and drift.

# 1. Generate estimates map
llm-cost export --format=json > estimates.json

# 2. Compare against actuals (with fuzzy matching)
llm-cost calibrate \
  --estimates estimates.json \
  --actuals billing-data.csv \
  --match fuzzy > factors.toml

Output factors.toml contains drift multipliers (e.g., 1.05 for +5% drift) and confidence scores.

FinOps Export

Export FOCUS 1.0 CSV for Vantage, CloudZero, or any FOCUS-compliant tool:

llm-cost export --format focus -o costs.csv

Filter and group by Tags.team, Tags.app, or Tags.model in your FinOps dashboard.

Commands

CommandPurpose
estimateCost estimate for prompt files (alias: price is deprecated)
countToken count only
checkBudget/policy enforcement
diffCost comparison between git refs
calibrateDrift analysis vs actual billing data
exportFOCUS CSV for FinOps tools
pipeStream JSON usage → cost output
update-dbRefresh pricing database

Note: Global flags (--quiet, --verbose) can be placed anywhere in the command.

How It Works

  • Offline: No API keys, no telemetry. Network only for explicit update-db.
  • Exact: BPE tokenizer with tiktoken parity (o200k_base, cl100k_base).
  • Signed: Pricing updates verified via Ed25519/TUF-lite manifest system.
  • Fast: ~10 MB/s throughput, O(N) complexity.

Security

  • SLSA Level 2 build provenance
  • Artifact attestations via Sigstore
  • Minisign-verified pricing database
  • Zero runtime network calls

Pricing Database

src/core/pricing/pricing_db.json contains standard rates.

Note: For Anthropic prompt caching, we default to the 5-minute cache write tier.

  • OpenAI: Updated to latest rates (see src/core/pricing/pricing_db.json).
  • Anthropic: Claude 3.5 Sonnet supports prompt caching rates.

🛡️ FinOps Certified (v1.3.0)

llm-cost is engineered for Enterprise FinOps. It goes beyond simple estimation to provide audit-grade validation.

  • P0/P1 Validation Suite: Every commit is verified against a rigorous regression test suite covering determinism, schema integrity, and scale (1M+ rows).
  • Cost Integrity Reporting: Automated "Cost Integrity Cards" in GitHub Pull Requests report drift (BPS), logic hashes, and PII leakage.
  • Fail-Fast Policy: Strict exit codes (2) ensures no "garbage-in/garbage-out" in data pipelines.
  • FOCUS Compliant: Native support for the FinOps Open Cost & Usage Specification (v1.0).
MetricStatus
Schema🟢 PASS
Logic🟢 PASS
Drift🟢 PASS

Verify releases:

gh attestation verify llm-cost-linux-amd64 --repo Rul1an/llm-cost

See docs/VERIFICATION.md.

Documentation

License

MIT

Ver o original no GitHub

Bom para

Relacionados