LangSmith CLI – Plugin para Claude Code
gigaverse-app/langsmith-cli
CLI minimalista para LangSmith com <100ms startup e 96% economia de contexto
Ver no GitHubCLI leve e eficiente em contexto para gerenciar projetos e execuções no LangSmith, empacotada como plugin do Claude Code. Oferece comandos otimizados para observabilidade, debugging e analytics de LLMs com 96% menos consumo de tokens que alternativas, incluindo monitoramento em tempo real, filtros avançados e exportação em múltiplos formatos.
Instalação
npx convez-ai gigaverse-app/langsmith-cli1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Observabilidade e debugging de pipelines LLM
- Análise de runs e métricas de performance em produção
- Filtragem avançada e sampling estratificado de traces
- Automação de fluxos de gerenciamento de projetos LangSmith
Veredito

Ferramenta sólida que resolve um problema real: substituir servidores MCP pesados por um CLI minimalista e eficiente em contexto. Ideal para agentes Claude trabalhando com LangSmith em produção. Maduridade técnica comprovada com badge de padrões Python modernos.
README
A tradução para o português deste README ainda está na fila.
🛠️ LangSmith CLI
The Modern CLI for LangSmith Lightning-fast • Context-efficient • Built for humans and AI agents
Features • Installation • Quick Start • Examples • Documentation
[!TIP] Install as Claude Code Plugin
Step 1: Install the CLI tool
# Quick install (Linux/macOS) curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh # Or use uv uv tool install langsmith-cliStep 2: Add the plugin to Claude Code
Option A: Terminal Command
claude plugin marketplace add gigaverse-app/langsmith-cli claude plugin install langsmith-cli@langsmith-cliOption B: Interactive UI Inside Claude Code, run
/plugin(without arguments) and use the UI to add the marketplaceThis gives Claude Code instant access to all LangSmith commands via skills.
🎯 Why LangSmith CLI?
Traditional tools are slow, verbose, and waste tokens. LangSmith CLI is different:
| Feature | LangSmith CLI | Official MCP Server |
|---|---|---|
| Startup Time | < 100ms (lazy loading) | ~2s (heavy imports) |
| Context Usage | 96% reduction with --fields | Full objects always |
| Human UX | Rich tables with colors | JSON only |
| Agent UX | Strict --json mode | Mixed output |
| Live Dashboard | runs watch real-time monitoring | ❌ |
| Advanced Filters | Regex, wildcards, time presets | Basic only |
| Stratified Sampling | runs sample by tag/metadata | ❌ |
| Aggregate Analytics | runs analyze group & metrics | ❌ |
| Tag Discovery | runs tags auto-discover patterns | ❌ |
| File Operations | View/analyze offline with globs | ❌ |
| Export Formats | JSON, CSV, YAML | JSON only |
100% Feature Parity + 10x Better QoL 🚀
✨ Features
🏎️ Performance First
- <100ms startup via lazy-loaded imports
- Streams large datasets without memory bloat
- Async-ready architecture
🧠 Agent Optimized
# Traditional: Returns 20KB trace object (1000+ tokens)
langsmith-cli runs get abc123
# Agent Mode: Returns only what you need (40 tokens)
langsmith-cli --json runs get abc123 --fields inputs,outputs,error
96% context savings on large traces!
🎨 Human Friendly
- Beautiful Rich tables with syntax highlighting
- Color-coded statuses (🟢 success, 🔴 error, 🟡 pending)
- Smart column truncation for readability
- Export to CSV/YAML for spreadsheets
🔍 Power User Features
# Regex filtering
langsmith-cli runs list --name-regex "^prod-.*-v[0-9]+"
# Wildcard patterns
langsmith-cli runs list --name-pattern "*auth*"
# Smart filters
langsmith-cli runs list --slow --failed --today
# Live dashboard
langsmith-cli runs watch
📦 Complete Coverage
Every LangSmith resource at your fingertips:
- ✅ Projects - List, create, inspect
- ✅ Runs - Search, stats, watch, sample, analyze, field discovery
- ✅ Datasets - CRUD + bulk JSONL uploads
- ✅ Examples - Full lifecycle management
- ✅ Prompts - Version control your prompts
- ✅ Self - Installation detection + auto-update
🚀 Installation
Quick Install (Recommended)
Linux/macOS:
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
Windows:
iwr -useb https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.ps1 | iex
This standalone installer:
- Creates an isolated environment (no conflicts)
- Automatically adds
langsmith-clito your PATH - Works without manually installing Python packages
- Requires Python 3.12+
Using uv
uv tool install langsmith-cli
Using pip
pip install langsmith-cli
For Claude Code Users
After installing the CLI above, add the skill:
/plugin marketplace add gigaverse-app/langsmith-cli
From Source
git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync
uv run langsmith-cli --help
🔑 Quick Start
1️⃣ Authenticate
langsmith-cli auth login
# Creates .env with your LANGSMITH_API_KEY
2️⃣ Explore Your Projects
langsmith-cli projects list
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Run Count ┃ Last Run ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ production │ 12,450 │ 2 mins ago │
│ staging │ 3,241 │ 15 mins ago │
│ development │ 892 │ 1 hour ago │
└───────────────┴────────────┴──────────────┘
3️⃣ Inspect Recent Runs
langsmith-cli runs list --project production --limit 5
4️⃣ Debug Errors Fast
langsmith-cli runs list --failed --recent --fields error
💡 Examples
🔍 Advanced Filtering
Find authentication runs that failed in the last hour:
langsmith-cli runs list \
--name-pattern "*auth*" \
--failed \
--recent
Search for specific versioned services:
langsmith-cli runs list \
--name-regex "^prod-api-v[0-9]+" \
--min-latency 5s \
--today
Multi-tag filtering (AND logic):
langsmith-cli runs list \
--tag production \
--tag experimental \
--slow
📊 Aggregated Insights
langsmith-cli runs stats --project production
{
"run_count": 12450,
"error_rate": 0.023,
"latency_p50": 0.234,
"latency_p99": 1.892,
"total_cost": 45.67,
"last_run_time": "2026-01-14T20:15:30Z"
}
🔴 Live Monitoring
langsmith-cli runs watch --project production
🔴 Live Dashboard (Ctrl+C to exit)
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Run ┃ Status ┃ Latency ┃ Time ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ ChatOpenAI │ 🟢 │ 0.234s │ Just now │
│ RetrievalChain │ 🟢 │ 1.456s │ 2s ago │
│ AuthService │ 🔴 │ 5.123s │ 5s ago │
│ WebScraper │ 🟡 │ - │ 8s ago │
└──────────────────┴────────┴──────────┴──────────┘
💾 Bulk Dataset Uploads
# Export examples to JSONL (using --output for reliable file writing)
langsmith-cli examples list --dataset my-dataset --output examples.jsonl
# Upload to new dataset
langsmith-cli datasets push examples.jsonl --dataset production-eval
🌐 Open in Browser
langsmith-cli runs open <run-id>
# Opens trace in LangSmith UI
📤 Export for Analysis
# Export to CSV for Excel
langsmith-cli runs list --format csv > runs.csv
# Export to YAML for configs
langsmith-cli projects list --format yaml > projects.yml
🔊 Verbosity Control
Control diagnostic output with industry-standard flags (following pip, Black, etc.):
# Default: Progress messages + warnings
langsmith-cli runs list --project production
# Quiet: Warnings only, no progress
langsmith-cli -q runs list --project production
# Silent: Errors only (cleanest for scripts)
langsmith-cli -qq runs list --project production
# Debug: Show API calls and processing details
langsmith-cli -v runs list --project production
# Trace: Ultra-verbose with HTTP requests and timing
langsmith-cli -vv runs list --project production
Clean JSON piping with -qq:
# Diagnostics on stderr, JSON on stdout
langsmith-cli --json runs list | jq
# Silent mode for cleanest piping
langsmith-cli --json -qq runs list | jq '.[] | .name'
# Suppress diagnostics with stderr redirection
langsmith-cli --json runs list 2>/dev/null | jq
🤖 AI Agent Integration
As a Claude Code Skill
[!NOTE] Prerequisites: Install CLI first (
uv tool install langsmith-cli), then add skill (/plugin marketplace add gigaverse-app/langsmith-cli)
The CLI is optimized for Claude Code agents:
# In Claude Code, agents can run:
langsmith-cli --json runs list --project default --limit 10 --fields inputs,outputs,error
# Returns clean JSON without Rich formatting
# Uses --fields to minimize context usage
# Filters with --failed, --slow, --recent for targeted debugging
Key Agent Patterns:
- Always use
--jsonas first argument - Use
--fieldsto reduce tokens by 90%+ - Combine smart filters:
--failed --recent --slow - Keep
--limitsmall (default: 10)
Example Agent Usage
import subprocess
import json
# Agent efficiently fetches only errored runs
result = subprocess.run(
["langsmith-cli", "--json", "runs", "list",
"--failed", "--limit", "5", "--fields", "error,inputs"],
capture_output=True,
text=True
)
runs = json.loads(result.stdout)
# Analyze errors with minimal context usage
📚 Documentation
Command Reference
langsmith-cli --help
# Core Commands
auth login # Authenticate with LangSmith
projects list # List all projects
runs list # Search and filter runs
runs get <id> # Inspect a specific run
runs get-latest # Get most recent run matching filters
runs stats # Aggregate statistics
runs watch # Live run dashboard
runs open <id> # Open trace in browser
runs search # Full-text search across runs
runs sample # Stratified sampling by tags/metadata
runs analyze # Group runs and compute metrics
runs tags # Discover tag patterns
runs metadata-keys # Discover metadata keys
runs fields # Discover field paths and types
runs describe # Detailed field statistics
runs view-file # View runs from JSONL files
runs usage # Token usage analysis with grouping
runs pricing # Model pricing coverage check
runs cache download # Download runs to local JSONL cache
runs cache list # List cached projects
runs cache clear # Clear cached data
datasets list # List datasets
datasets create # Create new dataset
datasets push # Bulk upload from JSONL
examples list # List dataset examples
examples create # Add example to dataset
prompts list # List prompt repositories
prompts get # Pull a prompt template
prompts push # Push local prompt to LangSmith
self detect # Show installation details
self update # Update to latest version
Global Flags
--json # Machine-readable JSON output (no Rich formatting)
--format {json|csv|yaml} # Export format
--help # Show help
--version # Show version
Runs Filtering
| Option | Description | Example |
|---|---|---|
--project | Filter by project name | --project production |
--project-id | Filter by project UUID | --project-id abc-123... |
--status | Filter by status | --status error |
--failed | Only failed runs | --failed |
--succeeded | Only successful runs | --succeeded |
--slow | Runs >5s latency | --slow |
--recent | Last hour | --recent |
--today | Today's runs | --today |
--min-latency | Min latency | --min-latency 2s |
--max-latency | Max latency | --max-latency 10s |
--since | Since time | --since "1 hour ago" |
--last | Last duration (or window with --since) | --last 24h or --since 2026-02-17 --last 72h |
--tag | Filter by tag (repeatable) | --tag prod --tag beta |
--name-pattern | Wildcard match | --name-pattern "*auth*" |
--name-regex | Regex match | --name-regex "^prod-.*" |
--model | Filter by model | --model gpt-4 |
🏗️ Architecture
Design Principles
1. Lazy Loading Performance
# ❌ Don't import heavy libraries at top level
from langsmith import Client
# ✅ Import inside command functions
@runs.command("list")
def list_runs(...):
import langsmith # Only loads when command executes
client = langsmith.Client()
2. Context Efficiency
- Default JSON output is "sparse" (only essential fields)
- Field pruning with
--fieldsfor targeted data extraction - No full trace blobs unless explicitly requested
3. Dual UX Pattern
# Check JSON flag for output mode
if ctx.obj.get("json"):
click.echo(json.dumps(data)) # Strict JSON for agents
else:
console.print(rich_table) # Beautiful tables for humans
4. DRY Utilities
- Shared helpers in
utils.py(100% test coverage) - Consistent error handling with field context
- Reusable filtering, sorting, and formatting functions
🧪 Development
Setup
# Clone and setup
git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync
# Install pre-commit hooks
uv run pre-commit install
Testing
# Run all tests
uv run pytest
# With coverage
uv run pytest --cov=src --cov-report=term-missing
# E2E tests (requires LANGSMITH_API_KEY)
export LANGSMITH_API_KEY="lsv2_..."
uv run pytest tests/test_e2e.py -v
Code Quality
# Linting and formatting
uv run ruff check --fix
uv run ruff format
# Type checking
uv run pyright
Project Stats
- 92% Test Coverage (589 tests)
- Zero Type Errors (Pyright clean)
- 100% MCP Parity (13/13 tools)
🤝 Contributing
Contributions welcome! Please:
- Fork the repository
- Create a feature branch (
git checkout -b feature/awesome) - Make your changes with tests
- Ensure 100% coverage for new code
- Run
uv run pre-commit run --all-files - Submit a pull request
📝 License
MIT License - see LICENSE for details.
🙏 Acknowledgments
Built with:
- Click - CLI framework
- Rich - Terminal formatting
- LangSmith SDK - Official Python client
- uv - Fast Python package installer
Made with ❤️ for the LangChain community