Agent Governance Research
Wiktor-Potapczyk/agent-governance-research
Governança e confiabilidade para agentes IA multi-tarefas
Ver no GitHubPesquisa empírica sobre padrões de governança para sistemas multi-agentes com LLMs, focando em classificação confiável, roteamento e execução de tarefas complexas. Inclui experimentos, benchmarks e arquitetura de frameworks desenvolvidos em produção com Claude Code.
Instalação
npx convez-ai Wiktor-Potapczyk/agent-governance-research1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Reduzir erros de classificação e roteamento em sistemas multi-agentes
- Avaliar mecanismos de compliance em agentes LLM
- Otimizar arquitetura de execução recursiva para tarefas compostas
- Implementar telemetria e observabilidade em agentes produtivos
Veredito

Pesquisa sólida e prática sobre governança de agentes IA, com contribuições reais (redução de Type II errors via exploration prompting). Repositório é mais acadêmico-técnico que ferramenta plug-and-play — ideal para quem constrói sistemas de agentes em produção e precisa entender mecanismos de confiabilidade.
README
A tradução para o português deste README ainda está na fila.
Agent Governance Research
Research into governance patterns for multi-agent LLM systems — how to make AI agents reliably classify, route, and execute complex tasks.
This repository contains both empirical research (experiments, benchmarks, data) and architectural insights (patterns, theories, framework specs) developed through production use of Claude Code.
Featured Paper
Forced Depth Consideration Reduces Type II Errors in LLM Self-Classification Wiktor Potapczyk, 2026
LLM task classifiers systematically misroute prompts that look simple but require deeper processing. We test whether prepending a single question before the classification decision reduces this failure mode, and run a mechanism ablation to identify why.
Key findings:
- Open-ended exploration ("What's really going on here?") reduces Type II errors to 1.25%, significantly outperforming directed extraction at 3.12% (p < 0.001, Bonferroni-corrected)
- A content-free metacognitive instruction ("Think carefully") achieves 1.0% — not significantly different from exploration (p = 0.77) — suggesting the mechanism is forced attention to complexity, not open-ended framing specifically
- Structured yes/no detection catastrophically harms some models (Claude Haiku: 330% error increase)
- Qualitative analysis reveals complementary failure modes: exploration forces committed implication statements; the metacognitive directive catches different trap subtypes via consequence framing
Paper (PDF + LaTeX), benchmark, and all experimental data: experiments/exploration-prompting-paper/
Research Threads
The full research index tracks 9 active threads: INDEX.md
| Thread | Summary |
|---|---|
| Exploration Prompting | Step-0 framing for self-classification (paper above) |
| Compound Task Neural Network | Tasks as continuous mixtures of 5 primitives |
| Quality Mechanism Spectrum | 8-level hierarchy from "think again" to ensemble. Extended 2026-05-25: Three-lens parallel ensemble applied to framework evaluation itself; convergent finding (confidence 0.82) — Wave A metadata improvements did not change any hook's detection behavior. Ensemble methodology documented as a repeatable pattern. |
| Inline Bias | Autoregressive generation bias in classification (12.4% mismatch) |
| Hooks as Governance | Hooks achieve ~90% compliance vs ~25% for prompt rules. Extended 2026-05-25: Infrastructure audit + multi-lens ensemble evaluation reveal the closed-loop measurement problem — internal compliance metrics measure rule conformance, not output quality. External calibration protocol (Action 0.1) is the only loop-exit channel. |
| Epistemic Gaps | Input (planning) and output (uncertainty) gaps |
| Framework Architecture | 5-node recursive execution pattern (2026-03-21). Extended 2026-05-25: cold-context empirical investigation of runtime state → 7-delta architecture spec closing gap between declared and actual behavior. Key finding: Stop hook event carries 11 hooks vs. 1–3 for all other events; enforcement is maximally rear-loaded. |
| Agent Teams & Depth Boundary | D1 governance constraint and collaborative mesh |
| Observability & Monitoring | 35-event telemetry catalog + 4-tier aggregation + 15 derivable conclusions |
2026-05 evolution: A major evaluation and re-architecture pass in May 2026 produced the documents in framework/, specs/, and the new meta/ batch. The central finding: the framework's instrumentation measures process conformance, not output quality — a closed-loop problem addressable only via external calibration. The architecture-v2 delta spec closes seven gaps between declared and runtime behavior. An ensemble methodology (parallel cold-context lenses, preserved divergent verdicts, main-session synthesis) is now used for evaluative work across the project. See specs/agent-governance-architecture-v2.md for the current architectural source of truth and specs/action-0-1-calibration-protocol.md for the external calibration protocol.
2026-06 addition: two insights generalize this layer's lessons (INDEX #40–#41) — the enforcement layer needs its own meta-verification (a BLOCK-hook/allow-list enforcer is itself code with silent false-positive and drift failure modes — verify the verifier), and installed is not adopted (a tool is adopted only when something pulls it into use at the right moment; incorporation and enforcement are independent axes). Subsequent June work added INDEX #42–#46: measure-then-gate (measure a failure rate before installing a blocking gate), wrong-protocol hooks silently no-op (the block format is event-specific), the workflow sub-agents have the full tool surface engine proof, the procedure-layer-as-workflows program spec (with two worked conversion drafts), and documenting an agent/skill/hook framework (the composition that became the framework repo's documentation standard). A 2026-06-10 curation pass indexed eighteen previously-unindexed worthwhile files (#47-#64), and the 2026-06-11 migration that converted the whole procedure layer to enforced workflow scripts produced two further insights (#65-#66): workflow-enforcement first-contact bugs (args-as-string, session-cached named invocation, write-restricted agent fabrication - and the gates holding through all of it) and Skill-tool-only hook assumptions (governance hooks silently break when a skill becomes a workflow; audit method + fix pattern).
Repository Structure
experiments/ # Empirical work
exploration-prompting-paper/ # Paper, benchmark, data, code
paper.pdf # Compiled paper
exploration-prompting.tex # LaTeX source
data/ # Raw results + computation scripts
classifiers/ # 8 classifier variant prompts
configs/ # Promptfoo configurations
framework/ # Architecture documentation
insights/ # Published research insights
theories/ # Theoretical work
patterns/ # Governance patterns
specs/ # Technical specifications
meta/ # Project metadata
Contributing
Looking for:
- Inter-rater validation: Label ~50 trap prompts as Quick vs. requires-deeper-processing (~1 hour). This directly addresses the paper's biggest methodological limitation. Prompts are in
experiments/exploration-prompting-paper/data/prompts-expanded.yaml. - Replication on open-weight models: All current data is from commercial APIs. Would the pattern hold on Llama, Mistral, Qwen?
- Methodological critique: What did we miss?
Citation
@article{potapczyk2026forced,
title={Forced Depth Consideration Reduces Type {II} Errors in {LLM} Self-Classification: Evidence from an Exploration Prompting Ablation Study},
author={Potapczyk, Wiktor},
year={2026},
note={Independent research. Code and data: \url{https://github.com/Wiktor-Potapczyk/agent-governance-research}}
}