convez.ai
PluginDevOps & InfraestruturaAvançado
William12556

LLM Governance and Orchestration

William12556/LLM-Governance-and-Orchestration

Governança estruturada e orquestração de LLMs com ciclos de trabalho auditáveis

Ver no GitHub
InicialPluginScore · de 0 a 100

Framework modelo-agnóstico para governança de LLMs em desenvolvimento de software, coordenando captura de requisitos, design e geração de código através de protocolos estruturados e gates de aprovação. Implementa o Ralph Loop (ciclo worker/reviewer) com orquestração autônoma limitada por governança explícita e rastreabilidade bidirecional entre requisitos, design, código e testes.

Instalação

$npx convez-ai William12556/LLM-Governance-and-Orchestration

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
2
Saúde
42%
Último commit
há 1 dia
Versão
v0.3.0
Issues abertas
0
Contribuidores
1

Casos de uso

  • Coordenação de projetos complexos com múltiplos modelos de IA
  • Garantia de rastreabilidade e conformidade em desenvolvimento assistido por IA
  • Decomposição estruturada de grandes projetos em tarefas discretas e auditáveis

Veredito

Plugin robusto para equipes que necessitam governança rigorosa em workflows de IA, com arquitetura dual-domain e protocolos bem definidos. Ideal para contextos críticos onde rastreabilidade e controle humano são não-negociáveis; requer investimento em integração e customização de perfis de implementação.

README

A tradução para o português deste README ainda está na fila.

LLM Governance and Orchestration

Purpose

This repository provides a model-agnostic governance framework for AI-assisted software development. The framework coordinates requirements capture, design, and code generation through structured protocols and human-in-the-loop approval gates.

For positioning context and design rationale, see RATIONALE.md.

The framework was motivated by a practical observation: language models lose coherence when navigating large, complex projects — a known consequence of context window constraints [1][2]. Structured documentation provides a compact, navigable project representation that mitigates this. The protocol-driven workflow achieves this by decomposing work into discrete, bounded steps — each providing the model with only the structured context relevant to the current task. Tactical Domain autonomy is explicitly bounded by governance protocols and a human-approved task brief, trading open-ended autonomy for predictability and traceability.

Governance

ai/governance.md defines a dual-domain architecture separating strategic coordination (Strategic Domain) from tactical implementation (Tactical Domain). Communication between domains uses MCP filesystem-based message passing. The framework is independent of any specific AI model or toolchain; implementation profiles map abstract framework concepts to concrete tooling.

  • Protocol-driven workflow: Eleven protocols (P00–P10) govern requirements capture, project initialization, three-tier design hierarchy, change management, issue resolution, traceability, testing, quality assurance, audit, prompting, and requirements management
  • Human approval gates: Explicit human authorization required before requirements baseline, design tier transitions, code generation, and baseline modifications
  • Three-tier design decomposition: Master (system) → Domain (functional) → Component (implementation) with validation gates between tiers
  • Model-agnostic architecture: Strategic and Tactical Domain roles fulfilled by any capable LLM; implementation profiles provided for Apple Silicon MLX (primary) and Claude Code (optional)
  • UUID-based document coupling: 8-character hex identifiers with iteration synchronization through debug cycles
  • Document lifecycle management: Active/closed states with immutable archival and closure criteria across all document classes
  • Bidirectional traceability: Requirements ↔ Design ↔ Code ↔ Test linkages
  • Template-based documentation: Seven YAML templates (T01–T07) for all document classes

Orchestration

The Autonomous Execution Loop (AEL) implements the Ralph Loop: a worker/reviewer cycle in which the same model fulfills both roles, differentiated by prompt engineering. The loop runs iteratively until the reviewer emits SHIP (task complete) or BLOCKED (boundary exceeded). Based on Geoffrey Huntley's Ralph Wiggum techniques.

orchestrator.py is an AI agent. It perceives its environment by reading state files and tool outputs, reasons via the model inference endpoint, acts by dispatching tool calls and writing state, and maintains persistent state in .ael/ralph/ across iterations. The Ralph Loop constitutes a minimal two-agent system: the worker agent produces, the reviewer agent critiques, and the orchestrator arbitrates. Autonomy is constrained by the governance protocols and the T04 tactical brief — the agent cannot redefine its goal mid-run.

orchestrator.py is the AEL entry point. It connects to configured MCP servers, sends tool definitions to the inference endpoint, dispatches tool calls, injects results, and iterates until no tool calls remain. It supports four execution modes:

ModeDescription
loopFull worker/reviewer Ralph Loop cycle (standard invocation)
workerSingle work phase pass
reviewerSingle review phase pass
resetClear state directory after human acceptance

Configuration — ai/ael/config.yaml:

SectionPurpose
omlxInference endpoint URL and default model
mcp_serversMCP server command and argument definitions
loopmax_iterations (outer Ralph cycles), phase_max_iterations (inner tool-call iterations per phase), MCP error threshold, tool call cap
contextModel directory path, context window size (or null to resolve from model config.json), warn/abort budget thresholds

State directory — .ael/ralph/ (ephemeral, per-task):

FileSignal
task.mdTask description loaded from T04 prompt
iteration.txtCurrent Ralph Loop cycle number
work-summary.txtWorker phase output
work-complete.txtWorker completion signal
review-result.txtSHIP or REVISE decision
review-feedback.txtReviewer notes for next worker iteration
.ralph-completeSuccess marker
RALPH-BLOCKED.mdFailure details; seeds T03 Issue

Invocation (from project root, after human approval of T04 prompt):

# Standard loop
python ai/ael/src/orchestrator.py --mode loop \
  --task workspace/prompt/prompt-<uuid>-<n>.md

# With wall-clock time limit (hours)
python ai/ael/src/orchestrator.py --mode loop \
  --task workspace/prompt/prompt-<uuid>-<n>.md \
  --duration 12
FlagPurpose
--modeworker | reviewer | loop | reset (default: loop)
--taskTask string or path to task file
--modelModel for all phases (overrides config default)
--worker-modelModel for work phase only (loop mode)
--reviewer-modelModel for review phase only (loop mode)
--max-iterationsOuter Ralph cycle limit override
--durationWall-clock time limit in hours (default: no limit)
--configPath to config.yaml

Audit Loop

The AEL includes a read-only codebase quality analysis capability. It uses dedicated recipes (audit-work.yaml, audit-review.yaml) in place of the standard Ralph recipes. The worker reads source files, records findings, and marks items in a traversal index. The reviewer checks finding quality and coverage. No source file is written.

Audit-specific state files (pre-populated by the Strategic Domain before launch):

FilePurpose
audit-index.mdOrdered list of items to audit; worker marks each [x] on completion
audit-report.mdAppend-only findings accumulator
audit-uml.mdOptional structural map of the target codebase

Audit criteria assessed per item: style, complexity, error handling, security, conformance, dead code.

Invocation:

python ai/ael/src/orchestrator.py --mode loop \
  --task workspace/prompt/<uuid>-audit.md \
  --duration 12

Without --duration the loop runs until all items in audit-index.md are marked complete or max_iterations is exhausted. High-severity findings are promoted to T03 issues post-run via the standard P04 workflow.

See docs/guide-audit-loop.md for an overview and framework/ai/doc/guide-audit-loop.md for operational detail.

govwatch — Governance Monitoring TUI

govwatch is a standalone read-only TUI for monitoring a downstream project's governance state. It scans workspace/ and .ael/ralph/ each polling cycle and reports:

  • Inferred workflow phase (Idle, Change cycle, Tactical execution, etc.)
  • Two-tier compliance alerts: coupling violations, UUID mismatches, invalid tactical_brief, naming convention failures
  • Open document registry grouped by UUID
  • Alert summary written to dashboard-alerts.md; clipboard copy via C key

Invocation from project root:

python ai/src/govwatch.py [--project PATH] [--interval N]

See framework/ai/doc/guide-govwatch.md for full operational detail.

ael-mcp — Claude Desktop AEL Interface (optional)

ael-mcp is a standalone MCP server that registers once in Claude Desktop and exposes three tools: start_ael, ael_status, and reset_ael. When registered, the Strategic Domain can launch and monitor AEL without human terminal access.

At T04 handoff (P09 §1.10.3), the human selects the execution path:

OptionWho launches AELStatus notification
A — Human executes (all profiles)Human runs terminal commandHuman notifies Strategic Domain
B — ael-mcp (Claude Desktop profile only)Strategic Domain calls start_aelStrategic Domain calls ael_status on request

Repository: https://github.com/William12556/ael-mcp
Setup instructions: P01 §1.2.8 in ai/governance.md

Repository Structure

DirectoryPurpose
framework/Canonical framework — governance, templates, profiles, AEL orchestrator
framework/ai/governance.mdMaster governance document
framework/ai/profiles/Implementation profiles (claude-desktop, claude, mlx)
framework/ai/templates/Document templates T01–T07
framework/ai/ael/Python AEL orchestrator (orchestrator, mcp_client, parser, recipes)
skel/Minimal deployable skeleton — copy to initialize a new project
skel/ai/Governance, templates, profiles, and recipes for deployed projects
skel/workspace/Pre-scaffolded workspace directories (design, prompt, test, trace, etc.)
skel/src/ skel/tests/ skel/docs/Standard project directories
dev/Framework development artefacts (requirements, design, issues, changes)
dev/requirements/Framework-level requirements documents
dev/design/Framework-level design documents
docs/User-facing operational documentation
docs/claude/Strategic Domain operational documents (instructions, guidelines, context)
docs/setup-apple-silicon-mlx.mdApple Silicon + MLX inference setup guide
docs/software-testing-guidance.mdSoftware testing reference for framework adopters
docs/guide-getting-started.mdEnd-to-end setup guide from clone to first AEL run
docs/guide-profile-selection.mdProfile selection guide: AEL vs Claude Code vs claude-omlx
docs/guide-audit-loop.mdAudit loop overview and usage guide
framework/ai/doc/guide-audit-loop.mdOperational audit loop guide for the Strategic Domain (downstream projects)
framework/ai/src/govwatch.pyGovernance monitoring TUI — read-only, downstream project observer
framework/ai/src/requirements-govwatch.txtgovwatch dependency manifest (textual, rich)
framework/ai/doc/guide-govwatch.mdgovwatch operational guide
framework/ai/doc/guide-ael-operations.mdAEL operational reference for the Strategic Domain (downstream projects)

Requirements

Common

ItemRequirement
Operating systemmacOS 14+ (Sonoma) required (Apple Silicon)
Python3.11+
GitAny recent version
Strategic DomainClaude Desktop (or equivalent frontier LLM with MCP support)
MCP serversFilesystem and mcp-grep configured in the Strategic Domain tool
govwatchpip install -r ai/src/requirements-govwatch.txt — required for governance monitoring TUI
Python MCP SDKpip install -r ai/ael/requirements.txt — required for AEL orchestrator
ael-mcpOptional — Claude Desktop AEL interface; clone from https://github.com/William12556/ael-mcp; requires mcp Python package

Apple Silicon + MLX

Required for the MLX inference backend (Tactical Domain on Apple Silicon).

ItemRequirement
ChipApple M-series (M1 or later)
Unified memory24 GB minimum (Q8); 48 GB+ for BF16
mlx_lm0.21+ — required dependency of oMLX (pip install mlx_lm)
omlxRequired inference server (pip install omlx)
ModelDevstral Small 2 (2512) Q8

Devstral Small 2 is the required Tactical Domain model. It is purpose-built for agentic coding and multi-file editing. The AEL parser (parser.py) is tuned to Mistral's tool-call format, which Devstral uses natively.

Full setup instructions: Apple Silicon + MLX Setup Guide


Getting Started

Prerequisites

  • MCP servers: Filesystem and mcp-grep configured in your Strategic Domain tool
  • Git and GitHub Desktop (or equivalent)
  • Tooling per selected implementation profile (see framework/ai/profiles/)

Initialization

  1. Copy skel/ to the desired parent directory and rename it to your project name
  2. Select an implementation profile from skel/ai/profiles/ and follow its setup instructions
  3. Ask your Strategic Domain model to read ai/governance.md and initialize the project per P01 (§1.2 Project Initialization)
  4. Begin with P00 (Governance) and follow the workflow flowchart in section 2.0

Implementation Profiles

ProfileTactical DomainAEL
claude.mdClaude Code (optional)Manual — human invokes per task
mlx_devstral_small_2_2512_Q8.mdMLX + Devstral Small 2 2512 (primary)AEL / Ralph Loop

Important Notice

This framework is experimental, serving as a learning exercise in prompt engineering, AI-assisted development workflows, and protocol-driven project management. Actual fitness for purpose is not guaranteed.


References

HUNTLEY, G., 2026. Everything is a ralph loop [online]. Available from: https://ghuntley.com/loop/ [Accessed 4 March 2026].

[1] FACTORY.AI, 2025. The Context Window Problem: Scaling Agents Beyond Token Limits [online]. Available from: https://factory.ai/news/context-window-problem [Accessed 20 March 2026].

[2] REDIS, 2026. LLM context windows: what they are & how they work [online]. Available from: https://redis.io/blog/llm-context-windows/ [Accessed 20 March 2026].

Version History

VersionDateDescription
1.02026-03-04Initial README; repository restructured to framework/ and skel/
1.12026-03-04Added Ralph Loop / Geoffrey Huntley attribution to Overview, AEL description, and References section
1.22026-03-04Renamed ai/implementation-profiles/ → ai/profiles/; renamed profile-*.md files to claude-desktop.md, claude.md, ollama.md
1.32026-03-05Expanded Repository Structure to reflect actual directory contents; removed duplicate Ralph Loop attribution from Overview; added framework/ai/doc/examples/ entry
1.42026-03-05Added Requirements section; created docs/ directory with setup guides for Goose, Apple Silicon + MLX, and OLLama + LM Studio
1.52026-03-05Added omlx as optional Apple Silicon + MLX requirement for TTL-based memory management
1.62026-03-06Added mlx.md to Implementation Profiles table
1.72026-03-06Promoted oMLX to required inference server; updated mlx_lm to dependency role
1.82026-03-11Replaced Goose with Python AEL orchestrator; updated repository structure, requirements, and implementation profiles table
1.92026-03-11Narrowed scope to Apple Silicon + MLX; deprecated Goose, OLLama, and LM Studio docs and profiles; moved to deprecated/
2.02026-03-12Added Devstral Small 2 (2512) as supported model in Requirements
2.12026-03-20Added motivation paragraph to Purpose; added references [1] and [2]
2.22026-03-20Extended motivation paragraph with workflow rationale
2.32026-03-26Revised Repository Structure: removed stale framework/ai/doc/examples/ and framework/ai/knowledge/ entries; added dev/, dev/requirements/, dev/design/, docs/claude/; updated skel/ description; added docs/ subdirectory entries
2.42026-03-27Replaced Overview and Key Characteristics with Governance and Orchestration sections; Orchestration covers AEL/Ralph Loop, orchestrator.py modes, config.yaml, state directory, and invocation
2.52026-03-31Added Devstral model rationale note to Apple Silicon + MLX Requirements
2.62026-03-31Updated Implementation Profiles table; deprecated mlx_devstral_small_2507_Q8.md; reinstated claude.md as optional Claude Code profile; updated model-agnostic architecture bullet
2.72026-04-28Added ael-mcp: Orchestration subsection (Option A/B launch table) and Requirements row
2.82026-05-20Added agent characterisation to Orchestration; added bounded autonomy rationale to Purpose; corrected copyright year
2.92026-05-20Added link to RATIONALE.md in Purpose section
3.02026-06-02Added Audit Loop subsection; added --duration flag and CLI flags table to Orchestration; updated Repository Structure with new guide documents
3.12026-06-10Added govwatch subsection; added govwatch entries to Repository Structure and Requirements

Copyright (c) 2026 William Watson. MIT License.

Ver o original no GitHub

Bom para

Relacionados