convez.ai
Agente
Uthmannabeel

warrant

Uthmannabeel/warrant

Ver no GitHub
InicialPluginScore · de 0 a 100

A licensing authority for AI agents - agents earn revocable, per-action licenses by making falsifiable predictions graded by reality. Built for the Splunk Agentic Ops Hackathon (Observability track).

Instalação

$npx convez-ai Uthmannabeel/warrant

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

README

A tradução para o português deste README ainda está na fila.

⚖️ Warrant

The licensing authority for AI agents.

License: MIT Python 3.11+ Track: Observability MCP

restart_connection_pool rollback_deploy clear_cache

Evals tell you how smart an agent is. Warrant tells production how much rope to give it — and takes the rope back the moment a prediction fails or the model changes underneath you.

🎬 Watch the 3-min demo · 🌐 Website · ▶ Live replay · 📜 Certificate · 📖 Docs

Watch the Warrant demo

▶ 3-minute demo on YouTube — or explore the live site.


Contents


The problem

Every vendor shipping agentic operations today says the same sentence: "human-led, with the analyst in control." It sounds responsible — but it quietly dodges the only question that decides whether any of this scales: when is the human allowed to let go?

In practice, autonomy gets granted the way it always does — a demo went well, a prompt was tweaked, someone flipped auto-approve. No number, no evidence, no way to take it back. That fails three silent ways: a lucky streak mistaken for competence, an agent that is confidently wrong, and the silent model swap that invalidates everything you thought you knew about it.

The idea

A driver's license doesn't certify that you're smart. It certifies you were tested on the road you'll actually drive — and it can be taken away.

Warrant treats AI autonomy exactly like a license. An agent earns a license per action class the way a pilot does: not by crashing real planes, but by passing graded exams in a proving ground. Every exam is judged against the agent's own falsifiable prediction — reality, not the model, decides if it was right. The license is revoked when a production prediction is violated, invalidated when the agent's brain changes, and decays if it goes unused.

How it works

The sandbox is live telemetry (a system you can deliberately break and heal). Splunk Cloud is the reasoning context, reached entirely through the Splunk MCP Server. And Warrant itself is an MCP server, so any external agent is gated by the same registry.

The control loop runs eight steps — and the keystone is that the prediction is committed before the action, so there is nothing to argue about at verification time:

1. SENSE      Read live telemetry                          (sandbox metrics)
2. CONTEXT    Pull real Splunk data + AI reasoning         (Splunk MCP: splunk_run_query, saia_*)
3. DIAGNOSE   A brain picks a bounded action + confidence  (heuristic or Gemini)
4. PREDICT    Commit to a FALSIFIABLE forecast band        (a control limit learned from healthy data)
              BEFORE acting   ← the keystone
5. GATE       Reversible + in-blast-radius, and act        (the safety kernel + the license)
              autonomously only with a valid LICENSE
6. ACT        Execute the bounded remediation              (sandbox control API)
7. VERIFY     Compare live metric vs. the forecast band    (read-back; optional late re-check)
8. LEDGER     Record the graded outcome (correct?,         (certification)
              confidence, brain fingerprint) → re-certify the license

What makes a license

A license for an action class is granted only when all of these hold:

ConditionWhy it matters
ConfidenceWilson score lower bound on the hit-rate ≥ threshold — one lucky pass can't license an action
EvidenceA minimum number of graded outcomes behind it
CalibrationBrier score ≤ limit — a confidently-wrong agent fails even with a good hit-rate
FingerprintPinned to the brain (model id + prompt version) — a change forces re-certification
ProbationEach production failure under the current brain raises the evidence bar (+2 samples per strike) — a suspended agent can't retry exam suites until one gets lucky
MarginGraduated autonomy: a thin margin over threshold yields ALLOW_WITH_MONITORING (act, but page a human); full autonomy needs a comfortable margin
FreshnessEvidence is time-weighted (configurable half-life) — a license decays to human-in-the-loop unless renewed with fresh outcomes

The ledger is tamper-evident: every outcome is sha256-chained to the one before it (Ledger.verify_chain(), exposed as the warrant_verify_ledger MCP tool), and each record is labelled by evidence — measured (Warrant read the metric itself) vs self-reported (the agent's word) — so an auditor can see exactly how much of a license rests on what.

▶ See it run — four acts

The demo runs Warrant against the live fault-injection sandbox. The kill-shot is Act II: a decoy fools the agent exactly the way it would fool an engineer — and its own falsifiable prediction catches the mistake, rolls it back, and suspends the license.

  1. Proving ground — the agent sits ~15 manufactured incidents in seconds and earns a license per action class (PROVISIONAL → LICENSED), without touching production.
  2. Production — a real leak (licensed → acts autonomously, proven right), then a decoy (obvious fix is wrong → prediction violated → "I was wrong" → escalate → rollback recovers → license SUSPENDED).
  3. Model updated overnight — the brain's fingerprint changes; every license drops to PROVISIONAL. Nothing else in ops notices a silent model swap. Warrant does.
  4. Re-certify — fresh exams re-earn the licenses under the new brain.

📜 Issued certificates & badges

Most "AI governance" is a dashboard number. Warrant issues a document — an official, printable certificate for every license, bound to the tamper-evident ledger hash — plus shields-style badges you can embed in any repo (like the live ones at the top of this README).

  • Live: GET /certificate/<action_class> (printable) and GET /badge/<action_class>.svg, served from the running dashboard.
  • Static: the badges above are committed under web/badges/ and hosted on Vercel — always-up, no backend.

🔌 Built on Splunk

Warrant consumes the Splunk MCP Server for every read of Splunk data, and ships its own MCP server so the trust gate becomes infrastructure any agent can call.

Splunk capabilityRole in Warrant
MCP Server (client)The agent's eyes — the CONTEXT step runs splunk_run_query over _internal
AI Assistant (saia_*)Hosted-model SPL authoring (saia_generate_spl) for the context query
Warrant as an MCP serverExposes the trust gate (warrant_request_action, warrant_report_outcome, warrant_check_license, warrant_list_licenses, warrant_verify_ledger) so any agent — a SOAR playbook, Splunk's Triage / Guided-Response agents, a Claude agent — can be licensed over MCP
SPL-native ledgersplunk/trust_ledger.spl recomputes the Wilson + Brier licensing math as a saved search, so on a real tenant the registry is a Splunk dashboard

We prove the gate end-to-end in warrant.mcp_demo: an independent agent earns, uses, and loses autonomy purely over MCP — and a second brain is refused a license it never earned.

🚀 Quickstart

Prerequisites: Python 3.11+. Splunk is optional for the demo (the loop falls back to a direct MCP query, and the brain defaults to a deterministic heuristic so every run is reproducible).

# 1. install
py -3.11 -m venv .venv ; .\.venv\Scripts\Activate.ps1 ; pip install -r requirements.txt

# 2. (optional) confirm Splunk connectivity through MCP
python -m warrant.check_connection

# 3. start the sandbox  (terminal A)
python -m uvicorn sandbox.app:app --port 9000

# 4a. the full four-act story in the console  (terminal B)
$env:WARRANT_BRAIN="heuristic"; python -m warrant.demo

# 4b. ...or the live dashboard, then open http://localhost:8050
$env:WARRANT_BRAIN="heuristic"; python -m uvicorn warrant.dashboard:app --port 8050

# 4c. ...or prove the MCP gate (self-contained — starts its own server)
python -m warrant.mcp_demo

# 4d. ...or watch a license rot from disuse (trust decay)
python -m warrant.decay_demo

On macOS/Linux, swap the venv activation for source .venv/bin/activate and $env:X="y" for X=y.

🏭 Using Warrant in a real environment

The governance engine and its MCP interface are real and reusable as-is; putting Warrant in front of a real company's agents is wiring, not rebuilding. Two connection points:

  1. Your agent calls the gate — any MCP-speaking agent gains four tool calls; nothing is rewritten.
    warrant_request_action(action_class="restart_connection_pool", agent_fingerprint="<model:prompt id>")
    # → ALLOW · ALLOW_WITH_MONITORING · REQUIRE_APPROVAL
    
  2. Point verification at your real metrics — pass your own endpoint and the committed band; Warrant fetches and grades it itself, so the agent can't self-certify.
    warrant_report_outcome(action_class="restart_connection_pool",
                           metric_url="https://your-splunk/.../error_rate", upper_limit=0.01)
    

Action execution stays in your existing runbooks/SOAR — Warrant governs whether to act, not how. A production deployment would still add MCP authentication, a shared durable ledger (the SPL version points the way), and operational hardening. See Limitations & roadmap.

⚙️ Configuration

All knobs are environment variables (see .env.example):

VariablePurposeDefault
WARRANT_BRAINDiagnosis brain: heuristic (reproducible), gemini, or autoauto
GEMINI_API_KEYEnables the Gemini brain (same safety harness either way)unset
WARRANT_AUTONOMY_THRESHOLDWilson lower bound a license must clear0.5
WARRANT_AUTONOMY_MIN_SAMPLESMinimum graded outcomes before licensing4
WARRANT_CALIBRATION_MAXMaximum Brier score (confidently-wrong fails here)0.4
WARRANT_PROBATION_EXTRAExtra evidence required per production failure2
WARRANT_MONITORING_MARGINMargin below which a license is ALLOW_WITH_MONITORING0.10
WARRANT_TRUST_HALFLIFE_DAYSEvidence half-life for trust decay30
SPLUNK_MCP_URL / SPLUNK_TOKENSplunk MCP endpoint + encrypted token (audience mcp)see SETUP.md

📁 Repository layout

warrant/
  sandbox/      Parameterised microservice "flight simulator": severities + 3 reversible controls
  warrant/      loop · proving_ground · certification · ledger · brain · splunk_mcp · mcp_server
                · dashboard · demo · mcp_demo · decay_demo
  splunk/       SPL trust-ledger saved search (Wilson + Brier as a native search)
  web/          The website + static replay + license badges + screenshots (hosted on Vercel)
  tools/        capture_replay.py — regenerate the static replay from a real run
  docs/         architecture_diagram.md · demo-script.md · devpost.md
  SETUP.md      Splunk account + MCP install + connectivity test

⚠️ Limitations & roadmap

Warrant is honest about what a hackathon proof-of-concept can and cannot prove:

LimitationStatus / mitigation
"Production" is a sandbox flight-simulatorBy design — you don't test a fire alarm by burning a building. The architecture is unchanged against real infrastructure.
Exams come from the same generator the agent later facesSeverity/noise are parameterised so no two exams are identical; a real deployment needs a richer scenario library.
Exam outcomes are correlated, not i.i.d.The Wilson bound is therefore optimistic about effective sample size; thresholds are configurable and conservative.
Human approval is simulated by defaultThe loop logs it as simulated and exposes LoopParams.approve for a real approval queue.
Verification reads one metric over a fixed horizonLoopParams.recheck adds a second, later reading; multi-signal verification is roadmap.
The local JSON ledger is hash-chained but file-writableTamper-evident, not tamper-proof. The SPL ledger is the production answer: the registry lives in Splunk.
Self-reported MCP outcomes can lieMitigated: measured mode (metric_url) lets Warrant grade outcomes itself; every record is labelled measured vs self-reported.
No auth on the MCP server yetFine for a demo; production needs caller authentication. On the roadmap.
Thresholds & decay half-life are demo-scaled constantsReal certification wants hundreds of graded outcomes; all knobs are env-configurable.

📄 License

Released under the MIT License — see LICENSE.

MIT License

Copyright (c) 2026 Warrant contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Ver o original no GitHub

Bom para

Relacionados