convez.ai
MCP ServerDevOps & InfraestruturaIntermediário
gigaverse-app

LangSmith CLI – Plugin para Claude Code

gigaverse-app/langsmith-cli

CLI minimalista para LangSmith com <100ms startup e 96% economia de contexto

Ver no GitHub
InicialPluginScore · de 0 a 100

CLI leve e eficiente em contexto para gerenciar projetos e execuções no LangSmith, empacotada como plugin do Claude Code. Oferece comandos otimizados para observabilidade, debugging e analytics de LLMs com 96% menos consumo de tokens que alternativas, incluindo monitoramento em tempo real, filtros avançados e exportação em múltiplos formatos.

Instalação

$npx convez-ai gigaverse-app/langsmith-cli

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
9
Forks
1
Acompanhando
0
Saúde
50%
Último commit
há 3 dias
Versão
v0.12.1
Issues abertas
43
Contribuidores
2

Casos de uso

  • Observabilidade e debugging de pipelines LLM
  • Análise de runs e métricas de performance em produção
  • Filtragem avançada e sampling estratificado de traces
  • Automação de fluxos de gerenciamento de projetos LangSmith

Veredito

Ferramenta sólida que resolve um problema real: substituir servidores MCP pesados por um CLI minimalista e eficiente em contexto. Ideal para agentes Claude trabalhando com LangSmith em produção. Maduridade técnica comprovada com badge de padrões Python modernos.

README

A tradução para o português deste README ainda está na fila.

🛠️ LangSmith CLI

The Modern CLI for LangSmith Lightning-fast • Context-efficient • Built for humans and AI agents

uv Ruff Python 3.12+ License: MIT

Features • Installation • Quick Start • Examples • Documentation


[!TIP] Install as Claude Code Plugin

Step 1: Install the CLI tool

# Quick install (Linux/macOS)
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

# Or use uv
uv tool install langsmith-cli

Step 2: Add the plugin to Claude Code

Option A: Terminal Command

claude plugin marketplace add gigaverse-app/langsmith-cli
claude plugin install langsmith-cli@langsmith-cli

Option B: Interactive UI Inside Claude Code, run /plugin (without arguments) and use the UI to add the marketplace

This gives Claude Code instant access to all LangSmith commands via skills.


context is all you need

🎯 Why LangSmith CLI?

Traditional tools are slow, verbose, and waste tokens. LangSmith CLI is different:

FeatureLangSmith CLIOfficial MCP Server
Startup Time< 100ms (lazy loading)~2s (heavy imports)
Context Usage96% reduction with --fieldsFull objects always
Human UXRich tables with colorsJSON only
Agent UXStrict --json modeMixed output
Live Dashboardruns watch real-time monitoring❌
Advanced FiltersRegex, wildcards, time presetsBasic only
Stratified Samplingruns sample by tag/metadata❌
Aggregate Analyticsruns analyze group & metrics❌
Tag Discoveryruns tags auto-discover patterns❌
File OperationsView/analyze offline with globs❌
Export FormatsJSON, CSV, YAMLJSON only

100% Feature Parity + 10x Better QoL 🚀


✨ Features

🏎️ Performance First

  • <100ms startup via lazy-loaded imports
  • Streams large datasets without memory bloat
  • Async-ready architecture

🧠 Agent Optimized

# Traditional: Returns 20KB trace object (1000+ tokens)
langsmith-cli runs get abc123

# Agent Mode: Returns only what you need (40 tokens)
langsmith-cli --json runs get abc123 --fields inputs,outputs,error

96% context savings on large traces!

🎨 Human Friendly

  • Beautiful Rich tables with syntax highlighting
  • Color-coded statuses (🟢 success, 🔴 error, 🟡 pending)
  • Smart column truncation for readability
  • Export to CSV/YAML for spreadsheets

🔍 Power User Features

# Regex filtering
langsmith-cli runs list --name-regex "^prod-.*-v[0-9]+"

# Wildcard patterns
langsmith-cli runs list --name-pattern "*auth*"

# Smart filters
langsmith-cli runs list --slow --failed --today

# Live dashboard
langsmith-cli runs watch

📦 Complete Coverage

Every LangSmith resource at your fingertips:

  • ✅ Projects - List, create, inspect
  • ✅ Runs - Search, stats, watch, sample, analyze, field discovery
  • ✅ Datasets - CRUD + bulk JSONL uploads
  • ✅ Examples - Full lifecycle management
  • ✅ Prompts - Version control your prompts
  • ✅ Self - Installation detection + auto-update

🚀 Installation

Quick Install (Recommended)

Linux/macOS:

curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

Windows:

iwr -useb https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.ps1 | iex

This standalone installer:

  • Creates an isolated environment (no conflicts)
  • Automatically adds langsmith-cli to your PATH
  • Works without manually installing Python packages
  • Requires Python 3.12+

Using uv

uv tool install langsmith-cli

Using pip

pip install langsmith-cli

For Claude Code Users

After installing the CLI above, add the skill:

/plugin marketplace add gigaverse-app/langsmith-cli

From Source

git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync
uv run langsmith-cli --help

🔑 Quick Start

1️⃣ Authenticate

langsmith-cli auth login
# Creates .env with your LANGSMITH_API_KEY

2️⃣ Explore Your Projects

langsmith-cli projects list
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project       ┃ Run Count  ┃ Last Run     ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ production    │ 12,450     │ 2 mins ago   │
│ staging       │ 3,241      │ 15 mins ago  │
│ development   │ 892        │ 1 hour ago   │
└───────────────┴────────────┴──────────────┘

3️⃣ Inspect Recent Runs

langsmith-cli runs list --project production --limit 5

4️⃣ Debug Errors Fast

langsmith-cli runs list --failed --recent --fields error

💡 Examples

🔍 Advanced Filtering

Find authentication runs that failed in the last hour:

langsmith-cli runs list \
  --name-pattern "*auth*" \
  --failed \
  --recent

Search for specific versioned services:

langsmith-cli runs list \
  --name-regex "^prod-api-v[0-9]+" \
  --min-latency 5s \
  --today

Multi-tag filtering (AND logic):

langsmith-cli runs list \
  --tag production \
  --tag experimental \
  --slow

📊 Aggregated Insights

langsmith-cli runs stats --project production
{
  "run_count": 12450,
  "error_rate": 0.023,
  "latency_p50": 0.234,
  "latency_p99": 1.892,
  "total_cost": 45.67,
  "last_run_time": "2026-01-14T20:15:30Z"
}

🔴 Live Monitoring

langsmith-cli runs watch --project production
🔴 Live Dashboard (Ctrl+C to exit)

┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Run              ┃ Status ┃ Latency  ┃ Time     ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ ChatOpenAI       │ 🟢     │ 0.234s   │ Just now │
│ RetrievalChain   │ 🟢     │ 1.456s   │ 2s ago   │
│ AuthService      │ 🔴     │ 5.123s   │ 5s ago   │
│ WebScraper       │ 🟡     │ -        │ 8s ago   │
└──────────────────┴────────┴──────────┴──────────┘

💾 Bulk Dataset Uploads

# Export examples to JSONL (using --output for reliable file writing)
langsmith-cli examples list --dataset my-dataset --output examples.jsonl

# Upload to new dataset
langsmith-cli datasets push examples.jsonl --dataset production-eval

🌐 Open in Browser

langsmith-cli runs open <run-id>
# Opens trace in LangSmith UI

📤 Export for Analysis

# Export to CSV for Excel
langsmith-cli runs list --format csv > runs.csv

# Export to YAML for configs
langsmith-cli projects list --format yaml > projects.yml

🔊 Verbosity Control

Control diagnostic output with industry-standard flags (following pip, Black, etc.):

# Default: Progress messages + warnings
langsmith-cli runs list --project production

# Quiet: Warnings only, no progress
langsmith-cli -q runs list --project production

# Silent: Errors only (cleanest for scripts)
langsmith-cli -qq runs list --project production

# Debug: Show API calls and processing details
langsmith-cli -v runs list --project production

# Trace: Ultra-verbose with HTTP requests and timing
langsmith-cli -vv runs list --project production

Clean JSON piping with -qq:

# Diagnostics on stderr, JSON on stdout
langsmith-cli --json runs list | jq

# Silent mode for cleanest piping
langsmith-cli --json -qq runs list | jq '.[] | .name'

# Suppress diagnostics with stderr redirection
langsmith-cli --json runs list 2>/dev/null | jq

🤖 AI Agent Integration

As a Claude Code Skill

[!NOTE] Prerequisites: Install CLI first (uv tool install langsmith-cli), then add skill (/plugin marketplace add gigaverse-app/langsmith-cli)

The CLI is optimized for Claude Code agents:

# In Claude Code, agents can run:
langsmith-cli --json runs list --project default --limit 10 --fields inputs,outputs,error

# Returns clean JSON without Rich formatting
# Uses --fields to minimize context usage
# Filters with --failed, --slow, --recent for targeted debugging

Key Agent Patterns:

  • Always use --json as first argument
  • Use --fields to reduce tokens by 90%+
  • Combine smart filters: --failed --recent --slow
  • Keep --limit small (default: 10)

Example Agent Usage

import subprocess
import json

# Agent efficiently fetches only errored runs
result = subprocess.run(
    ["langsmith-cli", "--json", "runs", "list",
     "--failed", "--limit", "5", "--fields", "error,inputs"],
    capture_output=True,
    text=True
)

runs = json.loads(result.stdout)
# Analyze errors with minimal context usage

📚 Documentation

Command Reference

langsmith-cli --help

# Core Commands
auth login              # Authenticate with LangSmith
projects list           # List all projects
runs list              # Search and filter runs
runs get <id>          # Inspect a specific run
runs get-latest        # Get most recent run matching filters
runs stats             # Aggregate statistics
runs watch             # Live run dashboard
runs open <id>         # Open trace in browser
runs search            # Full-text search across runs
runs sample            # Stratified sampling by tags/metadata
runs analyze           # Group runs and compute metrics
runs tags              # Discover tag patterns
runs metadata-keys     # Discover metadata keys
runs fields            # Discover field paths and types
runs describe          # Detailed field statistics
runs view-file         # View runs from JSONL files
runs usage             # Token usage analysis with grouping
runs pricing           # Model pricing coverage check
runs cache download    # Download runs to local JSONL cache
runs cache list        # List cached projects
runs cache clear       # Clear cached data
datasets list          # List datasets
datasets create        # Create new dataset
datasets push          # Bulk upload from JSONL
examples list          # List dataset examples
examples create        # Add example to dataset
prompts list           # List prompt repositories
prompts get            # Pull a prompt template
prompts push           # Push local prompt to LangSmith
self detect            # Show installation details
self update            # Update to latest version

Global Flags

--json              # Machine-readable JSON output (no Rich formatting)
--format {json|csv|yaml}  # Export format
--help              # Show help
--version           # Show version

Runs Filtering

OptionDescriptionExample
--projectFilter by project name--project production
--project-idFilter by project UUID--project-id abc-123...
--statusFilter by status--status error
--failedOnly failed runs--failed
--succeededOnly successful runs--succeeded
--slowRuns >5s latency--slow
--recentLast hour--recent
--todayToday's runs--today
--min-latencyMin latency--min-latency 2s
--max-latencyMax latency--max-latency 10s
--sinceSince time--since "1 hour ago"
--lastLast duration (or window with --since)--last 24h or --since 2026-02-17 --last 72h
--tagFilter by tag (repeatable)--tag prod --tag beta
--name-patternWildcard match--name-pattern "*auth*"
--name-regexRegex match--name-regex "^prod-.*"
--modelFilter by model--model gpt-4

🏗️ Architecture

Design Principles

1. Lazy Loading Performance

# ❌ Don't import heavy libraries at top level
from langsmith import Client

# ✅ Import inside command functions
@runs.command("list")
def list_runs(...):
    import langsmith  # Only loads when command executes
    client = langsmith.Client()

2. Context Efficiency

  • Default JSON output is "sparse" (only essential fields)
  • Field pruning with --fields for targeted data extraction
  • No full trace blobs unless explicitly requested

3. Dual UX Pattern

# Check JSON flag for output mode
if ctx.obj.get("json"):
    click.echo(json.dumps(data))  # Strict JSON for agents
else:
    console.print(rich_table)      # Beautiful tables for humans

4. DRY Utilities

  • Shared helpers in utils.py (100% test coverage)
  • Consistent error handling with field context
  • Reusable filtering, sorting, and formatting functions

🧪 Development

Setup

# Clone and setup
git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync

# Install pre-commit hooks
uv run pre-commit install

Testing

# Run all tests
uv run pytest

# With coverage
uv run pytest --cov=src --cov-report=term-missing

# E2E tests (requires LANGSMITH_API_KEY)
export LANGSMITH_API_KEY="lsv2_..."
uv run pytest tests/test_e2e.py -v

Code Quality

# Linting and formatting
uv run ruff check --fix
uv run ruff format

# Type checking
uv run pyright

Project Stats

  • 92% Test Coverage (589 tests)
  • Zero Type Errors (Pyright clean)
  • 100% MCP Parity (13/13 tools)

🤝 Contributing

Contributions welcome! Please:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/awesome)
  3. Make your changes with tests
  4. Ensure 100% coverage for new code
  5. Run uv run pre-commit run --all-files
  6. Submit a pull request

📝 License

MIT License - see LICENSE for details.


🙏 Acknowledgments

Built with:

  • Click - CLI framework
  • Rich - Terminal formatting
  • LangSmith SDK - Official Python client
  • uv - Fast Python package installer

Made with ❤️ for the LangChain community

Report Bug • Request Feature • Documentation

Ver o original no GitHub

Bom para

Relacionados