convez.ai
PluginDevOps & InfraestruturaIntermediário
chrisgleissner

LLM Tools

chrisgleissner/llm-tools

Monitore uso de LLM e otimize capacidade entre provedores locais

Ver no GitHub
BomPluginScore · de 0 a 100

Conjunto de ferramentas CLI para monitorar capacidade, quotas e disponibilidade de provedores de LLM locais (Codex, Claude, Copilot, etc.). Oferece controle de uso, agendamento inteligente e rotação entre provedores para evitar desperdício e otimizar recursos.

Instalação

$npx convez-ai chrisgleissner/llm-tools

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
2
Forks
1
Acompanhando
0
Saúde
57%
Último commit
há 3 dias
Versão
1.4.0
Issues abertas
13
Contribuidores
2

Casos de uso

  • Verificar capacidade disponível antes de iniciar tarefas em LLM
  • Agendar execução de prompts quando provedor ficar disponível
  • Rotacionar entre múltiplos provedores para processos autônomos contínuos
  • Validar confiabilidade de suspend/resume antes de execuções noturnas

Veredito

Ferramentas práticas e bem pensadas para quem trabalha com múltiplos provedores de LLM em linha de comando. Resolve problema real de observabilidade e eficiência de recursos, com abordagem zero-config que integra com autenticações existentes.

README

A tradução para o português deste README ainda está na fila.

llm-tools

Build codecov License: Apache 2.0 Platform

llm-tools is a small set of command-line tools for staying on top of local LLM provider capacity: session windows, weekly limits, quotas, credit balances, cost budgets, and provider availability.

The goal is to make LLM CLI work more observable and less wasteful. You can see which providers still have capacity, avoid burning weekly limits blindly, and dispatch tasks as soon as a provider becomes usable again instead of leaving open session windows idle.

The tools are intentionally local- and CLI-first. Instead of introducing another authentication layer, they use the provider CLIs you already have installed and authenticated. Credentials stay with those tools, and normal use remains zero-config.

Supported providers include: Codex, Claude Code, GitHub Copilot, Kilo Code, MiniMax, and OpenCode.

Tools at a Glance

CommandUse it when you want to...
llm-usageCheck remaining LLM capacity before starting work.
llm-schedulerRun one prompt through one selected provider once that provider has usable capacity.
ralph-robinKeep autonomous work moving by rotating across providers instead of stopping at the first limit.
llm-sleep-soakProve suspend/resume is reliable on this machine before trusting unattended overnight runs.

Install

Install with pipx:

pipx install git+https://github.com/chrisgleissner/llm-tools.git

This puts the commands on your PATH and keeps the package in its own virtual environment. It also works on externally managed Python installations such as Debian, Ubuntu, and Homebrew Python.

If you do not have pipx yet:

python3 -m pip install --user pipx
python3 -m pipx ensurepath

On macOS, you can also use Homebrew:

brew install pipx

Open a new shell, then verify the commands are available:

command -v llm-usage
command -v llm-scheduler
command -v ralph-robin

Install from a Release Wheel

Each release ships a wheel ZIP archive. You can install the wheel directly:

pipx install https://github.com/chrisgleissner/llm-tools/releases/download/0.3.2/llm_tools-0.3.2-py3-none-any.whl

Install from a Local Checkout

From a cloned repository:

pipx install .

Or install into a virtual environment:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install .

You can also run the tools directly from a checkout:

./llm-usage
./llm-scheduler
./ralph-robin

Quick Start

Check current capacity:

llm-usage
llm-usage --watch 60

Run a prompt once a specific provider is ready:

llm-scheduler --provider codex --prompt-file task.md
llm-scheduler --provider kilo --prompt-file task.md
llm-scheduler --provider minimax --prompt-file task.md

Keep work moving across providers:

ralph-robin --prompt-file task.md

Follow the latest scheduler run:

tail -f ~/.cache/llm-tools/llm-scheduler/logs/latest/run.log
tail -f ~/.cache/llm-tools/llm-scheduler/logs/latest/attempt-1.out

Provider CLI Requirements

llm-tools drives the official command-line clients for each provider. Install and authenticate the CLI for each provider you want to use.

ProviderCLI binaryInstall
Claude Codeclaudeclaude.com/product/claude-code - npm install -g @anthropic-ai/claude-code
Codexcodexgithub.com/openai/codex - npm install -g @openai/codex
GitHub Copilotcopilotgithub.com/github/copilot-cli - npm install -g @github/copilot
Kilo Codekilokilo.ai - npm install -g @kilocode/cli
MiniMaxmmxplatform.minimax.io - npm install -g mmx-cli
OpenCodeopencodeopencode.ai - npm install -g opencode-ai

You do not need every provider CLI installed.

  • llm-usage reports unavailable providers as unavailable and still shows the rest.
  • llm-scheduler only needs the provider selected with --provider.
  • ralph-robin skips unavailable providers and rotates across the usable ones. Its default rotation is claude,codex,opencode; use --providers to change it.

Capacity Scopes

llm-tools calls every quota-like constraint a scope.

A scope is one capacity measure exposed by one provider. For example, Codex and Claude can expose 5h and weekly reset windows, while Kilo can expose a credit balance or monthly budget.

KindResets?ExamplesProviders
reset_windowyes5h, weekly, monthlyCodex, Claude, Copilot, MiniMax
balancenoKilo credit balance, GBP/USD/creditsKilo, OpenCode
budgetyesMonthly spend budgetKilo, OpenCode
ungatedn/aBYOK, local, ungated modeKilo, OpenCode
opaquen/aPrepaid gateway subscriptionconfigured via routes

llm-usage shows one row per scope. llm-scheduler and ralph-robin can gate on a specific scope with --scope.

Per-provider scope allow-lists:

ProviderSupported --scope values
Codexauto, 5h, weekly
Claude Codeauto, 5h, weekly
MiniMaxauto, 5h, weekly
GitHub Copilotauto, monthly
Kilo Codeauto, balance, budget, byok, ungated
OpenCodeauto, balance, budget, byok, ungated

opaque is for capacity that exists but cannot be measured before launch — most commonly a prepaid subscription on a gateway. It is selected by an explicit route ([routes.<id>] with capacity.policy = "opaque"); see Route Mode below. The canonical scope name is subscription so the table reads naturally when the cost is a fixed periodic charge.

Route Mode

The default rotation is over providers ([providers.*], --providers). When the same provider can serve several underlying models with different capacity and cost semantics, you also have access to a route rotation ([ralph].routes, [routes.<route_id>]). A route binds a launch provider, a model, a capacity policy, and a cost policy into a single schedulable unit.

A route is shaped like this:

[ralph]
# When this list is present, ralph-robin rotates over routes in this
# order. In its absence the legacy provider rotation is used.
routes = ["kilo-minimax-m3"]

[routes.kilo-minimax-m3]
provider     = "kilo"
model        = "minimax-m3"
allow_fallback = false

[routes.kilo-minimax-m3.capacity]
# Capacity policies:
#   provider        - read the provider's own snapshot (default)
#   provider_model  - same, but model-aware (claude / codex have model-specific buckets)
#   delegate        - launch this route's provider, read capacity from another provider
#   opaque          - capacity exists but cannot be measured before launch
#   ungated         - usable when the launch CLI is present
#   balance         - read the provider's balance scope
#   budget          - read the provider's budget scope
policy = "opaque"
scope  = "subscription"     # display name; defaults to "subscription"
label  = "MiniMax M3 via Kilo"

[routes.kilo-minimax-m3.cost]
# Cost policies (display only; never affect readiness):
#   included, fixed_subscription, metered_balance, metered_budget,
#   free, external, unknown
policy   = "fixed_subscription"
amount   = 20
currency = "USD"
period   = "monthly"

llm-usage then renders:

Provider   Model       Ready   Scope          Remaining         Guidance              Resets in
Kilo       MiniMax M3  yes     subscription   prepaid USD20/mo   ✓ usable              -

opaque rows never display a percentage, balance, or reset time. prepaid USD20/mo is the cost text; routes without a fixed_subscription cost render as not metered. There is never a progress bar on these rows.

delegate is the route-level successor to the legacy providers.<x>.capacity_provider setting. The provider-level setting still works (it is mapped to an implicit route with capacity.policy = "delegate"), so existing configs do not need to migrate.

The orchestrator's runtime context and prompt injection include the selected route_id and launch provider, so a handoff-style prompt does not stale-route to a different provider.

Configuration File

For shared settings across llm-usage, llm-scheduler, and ralph-robin, drop a TOML file at one of these locations (first match wins):

  1. $LLM_TOOLS_CONFIG (explicit path)
  2. $XDG_CONFIG_HOME/llm-tools/config.toml
  3. ~/.config/llm-tools/config.toml

A missing file is fine - the tools just use their built-in defaults. Parsed with the standard library's tomllib (Python 3.11+), so no extra dependency.

What wins when settings overlap: built-in defaults < config file < CLI flags. A flag you pass on the command line always beats the same key in the file.

The main thing the file lets you set is which model each provider should run, and what to do when that model's limit is used up:

# ~/.config/llm-tools/config.toml

[defaults]
# Order ralph-robin tries providers when --providers isn't given.
providers     = ["claude", "codex", "opencode"]
# Which capacity check to use. One of:
# auto | 5h | weekly | monthly | balance | budget | byok | ungated
scope         = "auto"
# Minimum quota left before a provider is considered usable.
min_remaining = 1

[providers.claude]
# Run `claude --model sonnet`; only run while Sonnet has capacity.
model          = "sonnet"
# What to do when Sonnet's limit is used up:
#   false -> skip claude and switch to the next provider
#   true  -> keep claude and let it pick another model
allow_fallback = false
# Optional: override [defaults].scope / min_remaining for just this provider.
#scope          = "weekly"
#min_remaining  = 5

[providers.codex]
model          = "spark"
allow_fallback = false

[providers.opencode]
# Tie opencode's availability/capacity to another provider's usage windows
# instead of its own. Use it when the opencode CLI is configured to run another
# provider's model (e.g. the MiniMax API), so opencode's own balance is
# irrelevant. ralph-robin then only routes to opencode while minimax has
# capacity, ranks even-burn on minimax's remaining, and suspends on minimax's
# reset — while still launching the opencode CLI.
capacity_provider = "minimax"

[ralph]                                # ralph-robin-only settings (override [defaults] above)
# One example key — see config.example.toml for the full list:
providers      = ["claude", "codex", "kilo"]

[scheduler]                            # llm-scheduler-only settings (override [defaults] above)
# One example key — see config.example.toml for the full list:
provider       = "claude"

A complete template with every supported key (all commented out) is shipped at config.example.toml in this repository. Copy it to one of the locations above and uncomment the lines you want to set.

When model is set, llm-scheduler and ralph-robin call the provider with --model NAME and only run while that model still has capacity. With allow_fallback = false (the default), the tool treats the provider as unavailable once that model's limit is used up and switches to the next provider. With allow_fallback = true, the tool keeps trying the provider but drops the model setting and lets the provider's CLI pick a different model. Pass --model NAME on the command line to override the file for a single run.

capacity_provider ties one provider's gating to another provider's usage windows. It is generic: any provider may borrow any other provider's windows (a single hop — a provider cannot reference itself or a provider that itself delegates). The borrowing provider's own CLI is still the one launched; only the availability/capacity reading is delegated. The motivating case is a CLI configured to run a different provider's model — for example OpenCode pointed at the MiniMax API, where OpenCode's own balance says nothing about whether a run will succeed. With capacity_provider = "minimax", ralph-robin only routes to OpenCode while MiniMax has capacity, ranks even-burn on MiniMax's remaining-per-day, and suspends until MiniMax's reset; the scope you gate on (e.g. 5h, weekly) is then validated against MiniMax's windows rather than OpenCode's.

Unknown sections or keys are rejected at load time so typos surface immediately.

llm-usage

Use llm-usage before starting work, in status lines, or in scripts that need a compact view of local LLM capacity.

llm-usage
llm-usage --json
llm-usage --watch 60
llm-usage --show-copilot-credits
llm-usage --show-source
llm-usage --statusline

Default output shows all supported providers, with one row per capacity scope:

LLM Usage · 13:03

Bars: █ available · ░ spent
Guidance: 5h rows forecast runout; weekly/monthly/budget rows compare remaining quota to time left.
          ✓ lasts until reset · ! empty before reset · × empty · ↑ headroom · = on pace · ↓ conserve

Provider   Model     Ready   Scope     Remaining         Guidance              Resets in
────────   ───────   ─────   ───────   ───────────────   ───────────────────   ──────────
Codex                yes     5h        90% █████████░    ✓ lasts until reset   4h 34m
                             weekly    34% ███░░░░░░░    ↓ conserve            5d 2h
           Spark             5h       100% ██████████    ✓ lasts until reset   4h 34m
                             weekly    96% █████████░    ↑ headroom            6d 1h

Claude               no      5h         0% ░░░░░░░░░░    × empty               36m
                             weekly    91% █████████░    ↑ headroom            5d
           Sonnet            weekly   100% ██████████    ↑ headroom            5d

Copilot              yes     monthly   36% ████░░░░░░    ↓ conserve            17d 11h

Kilo                 yes     balance   £12.40            ✓ funded              -
                             budget    62% ██████░░░░    ↑ headroom            17d 4h

The Model column only appears when a provider reports model-specific limits. These sub-rows sit under their provider's section: Codex surfaces its Spark model, and Claude surfaces per-model weekly limits (e.g. Sonnet) alongside the aggregate window. Model rows are informational - they are shown for visibility but do not gate scheduling, which always uses the provider's aggregate scopes.

How to read the table:

ColumnMeaning
ModelThe specific model a row reports on (e.g. Spark, Sonnet). Blank for a provider's aggregate rows. Only shown when model-specific data exists.
Readyyes means every blocking scope for that provider has usable capacity now. no means at least one scope must reset or recover.
ScopeThe capacity measure, such as 5h, weekly, monthly, balance, or budget.
RemainingRemaining percentage, balance, or an unmetered state such as byok, local, or ungated.
GuidanceFor 5h, whether current burn should last until reset. For weekly, monthly, and budget scopes, whether usage is ahead of or behind a linear pace.
Resets inRelative reset time. Non-reset scopes show -.

llm-usage Options

OptionPurpose
-j, --jsonPrint stable JSON with generated_at, codex, claude, copilot, kilo, opencode, and minimax.
-w, --watch SECONDSRefresh continuously.
-C, --show-copilot-creditsInclude Copilot AI credits when parseable.
-S, --show-sourceShow where each usage row came from.
-s, --hide-sourceHide the source column. This is the default.
-R, --show-remaining-timeShow burn-time estimates.
-r, --hide-remaining-timeHide burn-time estimates. This is the default.
-D, --show-daily-budgetShow the Guidance column. This is the default.
-d, --hide-daily-budgetHide the Guidance column.
-K, --show-codex-sparkShow Codex Spark rows.
-k, --hide-codex-sparkHide Codex Spark rows.
-M, --copilot-monthly-reset-offset-days DAYSDay offset from month start for Copilot monthly reset.
-p, --provider-parallelism NNumber of provider readers to run concurrently. Default: CPU cores; env: LLM_USAGE_PROVIDER_PARALLELISM.
-t, --statuslineRead Claude Code statusline JSON from stdin and cache it.
-l, --log-onlySample providers and append to the usage log without printing a table.
-n, --no-headerOmit the table header.
-h, --helpShow help.

llm-scheduler

Use llm-scheduler when you want one specific provider to run one specific prompt, but only after that provider has usable capacity.

It is useful for:

  • delayed launches
  • rate-limit-aware retries
  • tmux launches
  • wake scheduling
  • suspend-until-ready workflows

Basic Usage

llm-scheduler --provider codex --prompt-file task.md
llm-scheduler --provider claude --prompt "Continue the work in this repo until CI is green"
llm-scheduler --provider copilot --prompt-file task.md --retry-delays 60,180,600
llm-scheduler --provider kilo --prompt-file task.md
llm-scheduler --provider minimax --prompt-file task.md

Required form:

llm-scheduler --provider codex|claude|copilot|kilo|minimax (--prompt TEXT | --prompt-file FILE) [options]

Common Examples

Run after a specific local time:

llm-scheduler --provider codex --prompt-file task.md --at "23:05"

Run inside tmux:

llm-scheduler --provider codex --prompt-file task.md --tmux llm-work

Schedule a wake-up:

llm-scheduler --provider codex --prompt-file task.md --wake

Suspend until the selected provider is ready:

llm-scheduler --provider claude --prompt-file task.md --scope 5h --suspend-until-ready

Show the resolved plan without launching:

llm-scheduler --provider codex --prompt-file task.md --dry-run

Runtime Behavior

  • In an interactive terminal, the provider is launched directly and output is written to attempt-N.out.
  • In headless or non-terminal mode, the provider runs through a captured PTY.
  • In tmux mode, the provider runs inside the requested tmux session or window.
  • The scheduler exits after success, terminal failure, or retry exhaustion.

llm-scheduler Options

OptionPurpose
-P, --provider PROVIDERProvider: codex, claude, copilot, kilo, opencode, or minimax.
-p, --prompt TEXTPrompt text.
-f, --prompt-file FILERead prompt from FILE, preserving content.
-a, --at TIMEDelay launch until a date -d compatible local time.
-a, --not-before TIMEDo not launch before a date -d compatible local time.
-s, --scope auto|5h|weekly|monthly|balance|budget|byok|ungatedCapacity scope to gate on. Default: auto.
-W, --window SCOPEDeprecated alias for --scope.
-m, --min-remaining PERCENTMinimum remaining capacity required to launch. Default: 1.
-i, --poll-interval SECONDSUsage polling interval. Default: 60.
-u, --max-unavailable-wait SECONDSMaximum wait when usage cannot be measured. Default: 900. Use 0 to wait forever. Known reset times still wait for reset.
-r, --retry-delays LISTRetry delays. Default: 60,180,600.
-R, --no-retryDisable retries.
-C, --cwd DIRSet provider working directory.
-F, --freshLaunch a fresh foreground provider process. This is the default.
-H, --headlessForce non-interactive provider command and captured PTY.
-T, --tmux SESSION[:WINDOW]Run through tmux.
-e, --command-template TEMPLATEOverride provider syntax. Supports {provider}, {prompt}, {prompt_file}, and {cwd}. Parsed with Python shlex, not a shell.
-y, --auto-confirmAuto-confirm recognised safe trust prompts. This is the default.
-Y, --no-auto-confirmDisable safe auto-confirmation.
-I, --headless-idle-timeout SECONDSAbort headless fresh mode after no output progress. Default: 600. Use 0 to disable.
-Q, --headless-question-timeout SECONDSAbort headless fresh mode after question-like output stalls. Default: 30. Use 0 to disable.
-L, --log-dir DIRSet scheduler log root.
-O, --run-dir DIRWrite or resume a specific run directory.
-d, --dry-runResolve usage, timing, command plan, and logs without launching.
-k, --wakeEnable best-effort wake scheduling.
-U, --suspend-until-readySchedule a resumed run, enable wake, suspend the machine, and continue after wake.
-x, --wake-testPrint wake diagnostics without scheduling work.
-h, --helpShow help.

Default Provider Commands

ProviderInteractiveHeadless
Codexcodex -C <cwd> <prompt>codex exec -C <cwd> <prompt>
Claude Codeclaude <prompt>claude --print <prompt>
GitHub Copilotcopilot -C <cwd> -i <prompt>copilot -C <cwd> --prompt <prompt>
Kilo Codekilo run <prompt>kilo run --dir <cwd> <prompt>
OpenCodeopencodeopencode run --dir <cwd> <prompt>
MiniMaxmmxmmx run --auto -C <cwd> <prompt>

Interactive Kilo Code, OpenCode, and MiniMax take no working-directory flag — they inherit it from the launching process (the scheduler sets the subprocess cwd). Headless Kilo Code and OpenCode inject no permission-bypassing flag; whether an autonomous run may act without prompting is governed by each tool's own permission config.

The default Claude adapter uses your local Claude Code permission settings. To override Claude Code settings for one scheduler run:

llm-scheduler --provider claude --prompt-file task.md --command-template 'claude --permission-mode plan --print {prompt}'

ralph-robin

Use ralph-robin when the task matters more than which provider runs it.

It runs a Ralph loop: a persistent autonomous workflow that keeps going instead of stopping when one provider reaches a limit, stalls, or becomes temporarily unusable.

This makes it useful for long-running coding, repair, hardening, documentation, and investigation tasks where stopping at the first rate limit would waste time.

ralph-robin wraps llm-scheduler and rotates across configured providers. It can either:

  • keep using the current provider until it is exhausted
  • distribute work more evenly so provider limits burn down at a similar rate

Even burn-down is the default.

When Ralph selects Claude Code through the built-in adapter, it uses Claude's stream-json print mode and renders that event stream as readable stdout. Assistant text, tool calls, and tool results appear while the run is active.

Basic Usage

ralph-robin --prompt-file task.md
ralph-robin --prompt "Continue until tests pass"
ralph-robin --providers claude,codex,copilot,kilo,minimax --prompt-file task.md
ralph-robin --prompt-file task.md --tmux llm-work
ralph-robin --prompt-file task.md --dry-run

Example output:

ralph-robin --prompt-file /home/chris/dev/ralph.prompt.md
[16:22:47] ◆ ralph-robin: · logs: /home/chris/.cache/llm-tools/ralph-robin/logs/20260613-162247-ralph-robin-r67_na63
[16:22:47] ◆ ralph-robin: · usage claude: usable (5h 22% left, weekly 85% left) | codex: usable (5h 99% left, weekly 35% left)
[16:22:47] ◆ ralph-robin: ✓ selected claude (even-burn)
[16:22:52 claude] I'll begin the RALPH loop iteration following FAST-PATH STARTUP. Let me start by establishing current state.
[16:22:54 claude] Tool call: Bash
[16:22:54 claude] {
[16:22:54 claude]   "command": "git status && echo \"---LATEST COMMIT---\" && git log --oneline -5 && echo \"---BRANCH---\" && git branch --show-current",
[16:22:54 claude]   "description": "Check git status, branch, and recent commits"
[16:22:54 claude] }

Each relayed provider line is prefixed with [time provider] by default. This makes a quiet increment distinguishable from a wedged one and keeps the active provider visible.

Customize the prefix with --prefix:

ralph-robin --prompt-file task.md --prefix time,provider,usage
ralph-robin --prompt-file task.md --prefix none

ralph-robin Options

OptionPurpose
-P, --providers LISTSet comma-separated provider rotation. Values include claude, codex, copilot, kilo, opencode, and minimax.
-p, --prompt TEXTPrompt text passed to the selected provider.
-f, --prompt-file FILEPrompt file passed to the selected provider.
-s, --scope auto|5h|weekly|monthly|balance|budget|byok|ungatedCapacity scope to gate on. Default: auto.
-W, --window SCOPEDeprecated alias for --scope.
-m, --min-remaining PERCENTMinimum remaining capacity required to launch. Default: 1.
-i, --poll-interval SECONDSPoll interval passed to llm-scheduler. Default: 60.
-u, --max-unavailable-wait SECONDSBound inconclusive usage waits before optimistic launch. Default: 900; 0 waits forever.
-r, --retry-delays LISTRetry delays. Default: 60,180,600.
-R, --no-retryDisable retries.
-e, --even-burnSpread work to burn provider quota down evenly. Enabled by default.
-E, --no-even-burnKeep using the current provider until it is exhausted.
-n, --max-iterations NStop after N successful increments. Default 0 means no iteration cap. Use 1 for single-shot.
-D, --max-duration DStop once D of wall-clock time elapses, such as 24h, 90m, 30s, or seconds. Default: 24h. Use 0 to disable. Whichever limit is reached first wins.
-I, --min-iteration-seconds NMinimum runtime floor for successive increments. Default: 5; 0 disables.
-x, --prefix LISTComma-separated fields stamped on each relayed provider line. Fields: time, provider, usage. Default: time,provider. Use none or an empty value to disable.
-X, --prefix-usage-interval SRefresh interval in seconds for the cached usage prefix field. Default: 15. Use 0 to refresh every line.
-C, --cwd DIRSet provider working directory.
-F, --freshLaunch a fresh provider process through llm-scheduler.
-H, --headlessForce non-interactive provider command and captured PTY.
-T, --tmux SESSION[:WINDOW]Execute through tmux via llm-scheduler.
-g, --command-template TEMPLATEOverride provider syntax. Supports {provider}, {prompt}, {prompt_file}, and {cwd}.
-y, --auto-confirmAuto-confirm recognised safe trust prompts. This is the default.
-Y, --no-auto-confirmDisable safe auto-confirmation.
-q, --headless-idle-timeout SECONDSAbort headless fresh mode after no output progress. Default: 600. Use 0 to disable.
-Q, --headless-question-timeout SECONDSAbort headless fresh mode after question-like output stalls. Default: 30. Use 0 to disable.
-S, --state-file FILEStore current provider index. Default: ${XDG_CACHE_HOME:-$HOME/.cache}/llm-tools/ralph-robin/state.json.
-L, --log-dir DIRSet Ralph log directory. Default: ${XDG_CACHE_HOME:-$HOME/.cache}/llm-tools/ralph-robin/logs.
-k, --wakePass best-effort wake scheduling to llm-scheduler.
-U, --suspend-until-readySuspend even for the selected provider's own wait gates.
--watchdogArm a hardware watchdog across each machine suspend so a wedged resume reboots instead of hanging. See Reliable sleep/wake.
-d, --dry-runResolve rotation and usage state without submitting.
-h, --helpShow help.

The following options are passed through to llm-scheduler:

-s, --scope
-m, --min-remaining
-i, --poll-interval
-u, --max-unavailable-wait
-r, --retry-delays
-R, --no-retry
-C, --cwd
-F, --fresh
-H, --headless
-T, --tmux
-g, --command-template
-y, --auto-confirm
-Y, --no-auto-confirm
-q, --headless-idle-timeout
-Q, --headless-question-timeout

ralph-robin Behavior

ralph-robin owns the full rotation loop: provider selection, retries, waiting, suspend decisions, and handoff between providers.

Launch Mode

  • Starts provider increments in autonomous headless mode by default, even from an interactive terminal.
  • Re-evaluates provider capacity before each increment.
  • Submits the same prompt again after each increment, so long-running work can continue across provider boundaries.

Provider Selection

By default, ralph-robin uses even burn-down. When several providers are ready, it prefers the provider with the highest remaining pace-adjusted capacity.

The selector:

  • ranks reset-window scopes such as weekly
  • ranks budget scopes such as Kilo budget
  • treats balance and ungated scopes as usable, but not pace-rankable
  • assumes an unknown or stale reset has a full week available, so the provider can still be ranked instead of being skipped

Use --no-even-burn to keep using the current provider until it is exhausted.

Blocking and Recovery

ralph-robin does not stop just because every provider is currently blocked.

When no provider is usable, it waits or suspends until the rotation can recover. The loop ends only when one of these conditions is met:

  • a non-recoverable failure occurs
  • a degenerate instant-success streak is detected
  • --max-duration is reached
  • --max-iterations is reached

Output Handling

Provider output is streamed live.

On interactive terminals, ralph-robin highlights:

  • status lines
  • diffs
  • commands
  • warnings
  • errors

Colors are disabled automatically for non-TTY output, TERM=dumb, NO_COLOR, or LLM_USAGE_NO_COLOR.

Failure Handling

If usage cannot be measured, ralph-robin tries that provider before suspending.

If a provider exits with a scheduler autonomy abort, ralph-robin skips that provider for the current invocation and tries the next usable provider.

Suspend and Wake Behavior

When all providers are blocked, Ralph sets an RTC wake-up timer for the earliest known reset across the whole rotation, then resumes its own loop after wake.

If suspend infrastructure is unavailable, the lead time is too short, LLM_SCHEDULER_NO_ACTUAL_SUSPEND=1 is set, or --dry-run is used, Ralph falls back to an in-process wait (the machine stays awake — always correct, it just forgoes power saving).

This is different from:

llm-scheduler --suspend-until-ready

That scheduler mode wakes into one selected provider. Ralph wakes back into cross-provider rotation.

Reliable sleep/wake

An overnight Ralph run waits out provider reset windows by suspending the whole machine. That is only safe if the box reliably resumes, and only sensible if nothing else suspends it underneath the run. Ralph handles both, simply and portably.

It defers your OS auto-suspend while it works. Most desktops auto-suspend after some idle period (for example KDE PowerDevil, GNOME, or logind's IdleAction). That idle timer does not know a headless job is busy, so it can suspend the machine mid-iteration — with no wake armed, leaving it asleep until you touch it (and then possibly hanging on a flaky resume). For its whole run Ralph holds a logind idle inhibitor (systemd-inhibit --what=idle) so the OS will not auto-suspend out from under it. An idle inhibitor does not block an explicit suspend, so Ralph still controls its own deliberate, RTC-armed suspends. You do not have to change your auto-suspend settings.

You can still keep your normal auto-suspend timeout. If you prefer the belt-and-braces approach, set it longer than a typical Ralph session — but that is brittle for long runs, which is exactly why Ralph inhibits idle instead of relying on it.

Its own suspends are verified and recorded:

  • A wake is always armed before suspending. Ralph never suspends without first arming an RTC wake (via rtcwake -m no when it can, otherwise a systemd-run --user WakeSystem=true timer). If it cannot arm one, it does not suspend — it waits awake instead.
  • Wakes are verified by behaviour. After resume, Ralph checks how far the wall clock landed from the target. A wake that fires far from target (or not at all) is treated as unreliable, and Ralph stays awake for the rest of that run rather than risk repeating a bad cycle.
  • Suspend churn is capped. A minimum awake interval between suspends and an optional per-run cap stop flaky suspend/resume hardware from being cycled dozens of times unattended.
  • A durable ledger survives a wedged resume. Ralph writes an fsync'd marker before each suspend and another after resume. If a resume ever wedges the machine and it is hard-reset, the unfinished cycle is still on disk — Ralph (and the soak tool) warn about it on the next start.
  • --watchdog (opt-in) recovers a wedged resume. With --watchdog, Ralph arms a hardware watchdog across each suspend so a hung resume reboots the machine instead of hanging. This needs a usable /dev/watchdog whose timer keeps counting across S3 (a TCO/iTCO or IPMI watchdog, or RuntimeWatchdogSec= in systemd-system.conf); without one it is a logged no-op.

Portability. The suspend backend is feature-detected, not distro-specific: it works across systemd-based Linux and degrades to an awake wait where the tools are missing. The design is modular so a future macOS backend (caffeinate / pmset schedule wake) can be added without changing how Ralph uses it.

Tuning knobs: LLM_TOOLS_NO_INHIBIT, LLM_TOOLS_SUSPEND_DRIFT_TOLERANCE, LLM_RALPH_MIN_AWAKE_SECONDS, LLM_RALPH_MAX_SUSPENDS, LLM_SCHEDULER_SUSPEND_MIN_LEAD, LLM_TOOLS_WATCHDOG_DEVICE. Run llm-scheduler --wake-test to see what the current host supports.

llm-sleep-soak — prove sleep/wake is reliable

Before trusting unattended overnight runs, soak-test the exact suspend/wake path on your hardware:

llm-sleep-soak --cycles 50 --period 90s        # 50 real suspend/wake cycles
llm-sleep-soak --cycles 20 --period 2m --watchdog --json

Each cycle suspends the machine, wakes it via the same verified RTC path Ralph uses, measures the wake drift, scrapes the kernel log for resume errors, and records the cycle in the durable ledger. It prints a PASS/FAIL summary and exits non-zero if any cycle resumed late or logged a resume error — or if an earlier run left a cycle unfinished (the fingerprint of a past wedged resume).

This is a real-hardware test: it genuinely suspends the machine and therefore cannot run in CI. Run it when the machine is otherwise idle. LLM_SCHEDULER_NO_ACTUAL_SUSPEND=1 runs the whole loop in simulation (no real sleep) if you just want to see the flow.

Provider Setup Details

Most providers only need their CLI installed and authenticated once. Kilo and MiniMax also support environment-variable fallbacks, which are useful for CI and deterministic tests.

Kilo Code

Kilo is configured primarily through environment variables, so it can be driven from CI or Ralph without changing local state.

VariablePurpose
LLM_USAGE_KILO_MODEgateway (default), budget, byok, local, or ungated.
LLM_USAGE_KILO_BALANCERemaining credit balance. Required for balance scope.
LLM_USAGE_KILO_CURRENCYCurrency or unit label, such as GBP, USD, or credits.
LLM_USAGE_KILO_MIN_BALANCEMinimum remaining balance required to consider Kilo usable. Default: 1.
LLM_USAGE_KILO_MONTHLY_BUDGETTotal monthly budget. Enables budget scope.
LLM_USAGE_KILO_MONTHLY_SPENTAmount already spent in this budget period.
LLM_USAGE_KILO_MONTHLY_RESET_DAYDay of month the budget resets. Default: 1.

When kilo is on PATH, llm-usage and llm-scheduler try kilo stats first. JSON and text output are supported. If that fails or cannot be parsed, they fall back to the environment variables above.

With --scope auto, Kilo prefers:

  1. budget, when configured
  2. balance, when configured
  3. ungated

Gateway-backed models via routes (Kilo + MiniMax M3)

When Kilo sells a model from another provider (e.g. minimax-m3 purchased through the Kilo gateway), the entitlement lives behind the Kilo gateway: the direct mmx quota show reads the user's direct MiniMax account, not the Kilo-purchased subscription, so it is the wrong truth source. Model the route as opaque:

[ralph]
routes = ["kilo-minimax-m3"]

[routes.kilo-minimax-m3]
provider = "kilo"
model    = "minimax-m3"
[routes.kilo-minimax-m3.capacity]
policy = "opaque"
scope  = "subscription"
label  = "MiniMax M3 via Kilo"
[routes.kilo-minimax-m3.cost]
policy   = "fixed_subscription"
amount   = 20
currency = "USD"
period   = "monthly"

llm-usage then renders:

Provider   Model       Ready   Scope          Remaining         Guidance   Resets in
Kilo       MiniMax M3  yes     subscription   prepaid USD20/mo   ✓ usable   -

The route is usable whenever the Kilo CLI is on PATH and no local runtime block is recorded. If a Kilo run returns a real retryable error (e.g. HTTP 429, quota exceeded), the scheduler records a local block under ${XDG_CACHE_HOME:-$HOME/.cache}/llm-tools/routes/blocks/<id>.json so Ralph stops selecting the route until the retry window passes. A successful run clears the block.

The legacy providers.<x>.capacity_provider = "<y>" setting is for the truthful delegation case (the configured CLI runs another provider's model and that provider's windows truthfully describe the capacity). Use routes.<id>.capacity.policy = "opaque" when no such truth source exists.

MiniMax

MiniMax quota is read from the local mmx CLI first. Environment variables are available as a fallback for tests and controlled environments.

VariablePurpose
LLM_USAGE_MINIMAX_5H_PERCENTRemaining percentage for the 5h session window, from 0 to 100.
LLM_USAGE_MINIMAX_5H_RESET_EPOCHEpoch seconds or milliseconds when the 5h window resets.
LLM_USAGE_MINIMAX_WEEKLY_PERCENTRemaining percentage for the weekly window, from 0 to 100.
LLM_USAGE_MINIMAX_WEEKLY_RESET_EPOCHEpoch seconds or milliseconds when the weekly window resets.
LLM_USAGE_MINIMAX_MODELmodel_remains row to read. Default: general.
LLM_USAGE_MINIMAX_TIMEOUTTimeout for mmx quota show, in seconds. Default: 10.

When mmx is on PATH, llm-usage and llm-scheduler try:

mmx quota show --output json

If the CLI is missing or the output cannot be parsed, they fall back to the environment variables above.

The MiniMax reader uses the general row from model_remains by default and exposes the same 5h and weekly reset windows used by Claude Code and Codex. This lets the table render and gate MiniMax consistently with other reset-window providers.

The MiniMax row appears only when the mmx CLI is installed or MiniMax environment variables are set.

Logs and Cache

Runtime data lives under:

${XDG_CACHE_HOME:-$HOME/.cache}/llm-tools

Directory layout:

llm-tools/llm-usage/                 Usage caches and llm-usage.log
llm-tools/llm-scheduler/logs/        Per-run scheduler logs
llm-tools/ralph-robin/               Ralph state and logs

Each scheduler run directory contains:

run.log
events.jsonl
prompt.txt
attempt-N.out
attempt-N.status

The scheduler logs:

  • arguments
  • prompt source
  • prompt SHA-256
  • prompt content
  • usage snapshots
  • wait decisions
  • command plan
  • output
  • exit code
  • retry delays
  • final status

Useful symlinks:

~/.cache/llm-tools/llm-scheduler/logs/latest
~/.cache/llm-tools/llm-scheduler/logs/latest-claude
~/.cache/llm-tools/llm-scheduler/logs/latest-codex

Ralph logs live under:

${XDG_CACHE_HOME:-$HOME/.cache}/llm-tools/ralph-robin/logs

Child scheduler logs are written under each Ralph run's scheduler/ subdirectory.

Data Sources

llm-tools reads local provider state where possible.

ProviderSource
CodexLive codex app-server rate limits, then cache, then local ~/.codex/sessions JSONL.
Claude CodeOAuth usage API/cache with automatic OAuth token refresh, then statusline cache, then local project JSONL fallback.
GitHub CopilotLocal Copilot CLI footer captured through a bounded PTY helper.
Kilo Codekilo stats, then Kilo environment variables.
MiniMaxmmx quota show --output json, then MiniMax environment variables.

llm-usage reads providers concurrently. Configure fan-out with --provider-parallelism or LLM_USAGE_PROVIDER_PARALLELISM; the default is the number of CPU cores.

How refreshing works

Every provider is actively refreshed on each run - llm-usage never just echoes a session log left behind the last time you used a CLI. Each reader asks the provider for current numbers and only falls back if that fails:

ProviderActive refresh → fallback
Codexcodex app-server (live, turn-free) → last cached payload → local session JSONL
Claude CodeOAuth usage API (auto-refreshing the token) → API cache → statusline cache → project JSONL
GitHub CopilotBackground PTY footer capture → cached capture
Kilo / MiniMax / OpenCodekilo stats / mmx quota show / opencode stats → environment variables

A provider only reports stale-usage if it cannot be refreshed for a known authentication or CLI-startup reason - e.g. Codex shows not-authenticated (no ~/.codex/auth.json credentials) or missing-cli (the codex binary is not on PATH). When the CLI is installed and signed in, you always see live data.

The local-snapshot fallbacks (Codex/Claude session files) are still treated as stale after LLM_USAGE_LOCAL_SNAPSHOT_MAX_AGE seconds (default 60, capped at 60) while they claim an active window, so a brief network blip degrades to "unavailable" rather than to a misleadingly old percentage.

While readers are in flight, llm-usage shows a small spinner that erases itself once the table is ready. In --watch mode it docks to the right of the clock in the header (LLM Usage · 14:29 ⠙ refreshing usage 3/6) and the frame redraws in place - no full-screen wipe, so the dashboard updates without a flash. It uses only the most portable cursor sequences (ESC[H, ESC[K, ESC[r;cH, and ESC 7/ESC 8 save-restore), so it renders correctly under tmux, GNU screen, and a raw telnet PTY. Outside --watch, the spinner sits on stderr and self-erases. It is shown only on an interactive terminal, so pipes, --json consumers, and batch scripts stay byte-clean. Disable it with LLM_USAGE_NO_PROGRESS=1.

GitHub Copilot Notes

The Copilot footer only shows plan/session usage when the quota and ai-used status-line items are enabled. These are off by default on a fresh install and are normally toggled via /statusline.

Before each capture, llm-usage enables those items in:

${COPILOT_HOME:-~/.copilot}/settings.json

All other settings are preserved.

Set this variable to skip the settings write:

LLM_USAGE_COPILOT_NO_SETTINGS_WRITE=1

Copilot capture is cached with LLM_USAGE_COPILOT_CACHE_TTL. The PTY footer capture is slow and occasionally flaky, so when a refresh cannot complete within its wait budget, llm-usage shows the most recent monthly figure ("usage on start") and refreshes it in the background for the next run, rather than dropping to unavailable.

Default:

LLM_USAGE_COPILOT_CACHE_TTL=300

Force synchronous capture:

LLM_USAGE_COPILOT_CACHE_TTL=0

Wake Support

llm-scheduler --wake is best effort.

It prefers a transient user systemd-run timer with WakeSystem=true. If that is unavailable, it logs an rtcwake fallback command.

llm-scheduler --suspend-until-ready schedules a resumed scheduler invocation, then calls:

systemctl suspend

after the timer is accepted.

Configure the pre-suspend confirmation pause:

LLM_SCHEDULER_PRE_SUSPEND_CONFIRMATION_SECONDS=10

Wake reliability depends on:

  • firmware and BIOS/UEFI settings
  • motherboard RTC support
  • kernel support
  • systemd user timers
  • power state

The tool does not modify BIOS/UEFI settings and does not silently require sudo.

Run diagnostics with:

llm-scheduler --wake-test

Appearance and Output Customization

Color override example:

LLM_TOOLS_COLOR_ERROR='1;34'

Supported color roles:

BRAND, INFO, OK, WARN, ERROR, DIM, DIFF_ADD, DIFF_REMOVE, DIFF_HUNK,
COMMAND, TOOL, STDERR, HEADING

Symbol override examples:

LLM_TOOLS_SYMBOL_ERROR=!
LLM_TOOLS_NO_SYMBOLS=1

Ralph-launched provider processes inherit:

LLM_TOOLS_RALPH_ROBIN_ACTIVE=1
LLM_TOOLS_RALPH_ROBIN_SELECTED_PROVIDER
LLM_TOOLS_RALPH_ROBIN_PROVIDERS

If a child process tries to run llm-scheduler --suspend-until-ready while Ralph is active, the scheduler exits with status 75 instead of suspending. Ralph remains the single rotation and suspend coordinator.

Requirements

  • Linux or macOS.
  • Python 3.11 or newer.
  • Optional: tmux for tmux mode.
  • Optional: systemd-run or rtcwake for wake support.

Wake and suspend features require Linux with systemd:

--wake
--suspend-until-ready

Everything else works on Linux and macOS.

Limitations

  • Uses local data and locally authenticated CLIs only.
  • Not an official billing dashboard.
  • Missing or inconclusive provider data is shown as -, unknown, or unavailable.
  • If usage remains unavailable beyond --max-unavailable-wait, the scheduler launches optimistically and lets provider rate-limit handling and retry behavior take over.
  • Provider local data formats and CLI syntax can change.
  • Copilot AI credits are parsed when requested, but scheduler gating currently uses monthly remaining usage.

Adding a Provider

llm-tools uses a small provider-adapter contract. To add a new CLI, for example acme-cli:

  1. Add a provider module:

    llm_tools/providers/acme.py
    

    It should expose:

    read(env) -> ProviderSnapshot
    

    The snapshot carries zero or more CapacityScope objects.

  2. Use one of the supported capacity scope kinds:

    reset_window
    balance
    budget
    ungated
    unknown
    
  3. Re-export the module from:

    llm_tools/providers/__init__.py
    
  4. Register supported scopes in:

    llm_tools/capacity.PROVIDER_SCOPES
    

    Example:

    PROVIDER_SCOPES["acme"] = {SCOPE_AUTO, ...}
    
  5. Add default launch commands under:

    llm_tools.scheduler.provider_default_argv
    

    Add both attached and headless paths where applicable.

  6. Optionally add a highlighting pattern in:

    scheduler.highlight_provider_text
    
  7. Add --provider and --providers membership in the relevant validators:

    scheduler.py
    ralph_robin.py
    

After that, the generic decision logic in llm_tools/capacity.decide handles the provider's scopes. llm-usage, llm-scheduler, and ralph-robin can then use the new provider through the same model as the existing ones.

Tests

Install test dependencies:

python -m pip install -e . pytest coverage

Run tests with coverage:

coverage run -m pytest
coverage combine
coverage report --fail-under=85

Tests use fixtures and mock commands. They do not require:

  • real Codex, Claude, Copilot, Kilo, or MiniMax installations
  • provider credentials
  • network access
  • the user's real home directory

For manual end-to-end checks, run the examples above against installed and authenticated providers without the test fixture environment.

Contributing

Small, focused pull requests are welcome.

Before opening a PR, make sure total coverage is at or above 85%:

python -m pytest -q
coverage run -m pytest
coverage combine
coverage report --fail-under=85

License

Apache License 2.0.

Ver o original no GitHub

Bom para

Relacionados