convez.ai
AgenteDevOps & InfraestruturaAvançado
ld-singh

Agente de Revisão Arquitetural para SDLC

ld-singh/ai-sdlc-architecture-review-agent

Revisão arquitetural agentic com RAG para gating automático em PRs

Ver no GitHub
InicialPluginScore · de 0 a 100

Sistema multi-agente que realiza revisão arquitetural automatizada em pull requests, analisando impactos em arquitetura, CI/CD, segurança e operações. Gera um pacote de decisão com gate de aprovação humana e roda totalmente offline, integrando-se ao fluxo de desenvolvimento como porta automatizada de arquitetura.

Instalação

$npx convez-ai ld-singh/ai-sdlc-architecture-review-agent

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
1
Forks
0
Acompanhando
1
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Automatizar gates arquiteturais em CI/CD de times enterprise
  • Detectar riscos de arquitetura e design antes da produção
  • Gerar relatórios estruturados de impacto arquitetural e ADRs

Veredito

Implementação sólida de automação arquitetural com controle humano preservado. Reduz inconsistência em revisões de design mantendo auditoria e responsabilidade. Necessário expertise em setup, mas oferece ROI claro em times que entregam frequentemente.

README

A tradução para o português deste README ainda está na fila.

AI-SDLC Architecture Review Agent

Agentic, RAG-grounded architecture review for enterprise software delivery. A team of specialised agents reviews issues and pull request diffs, surfaces architecture and delivery risk with clear trade-offs, and hands a decision pack to a human who owns the call.

Python License Default Control PRs

AI-SDLC Architecture Review Agent

This is a working reference implementation of an agentic AI-SDLC capability: a multi-agent system that moves architecture review into the pull request, where design decisions are still cheap to change. It reads a GitHub issue or pull request, grounds itself in your repository and docs through retrieval (RAG), and reasons across architecture, CI/CD and release risk, security and governance, testing and operations to produce an architecture decision pack, with a human approval gate on anything risky.

The leverage is in delivery: it runs as a single command, so it drops into CI/CD as an automated architecture gate on every pull request. Architectural standards and ADR requirements become a repeatable, auditable step that scales across teams instead of depending on who reviews. High or critical risk blocks the merge until a named architect signs off; low-risk changes flow through. It runs fully offline by default, with no API key required.

The goal is not autonomous code generation. The goal is safer AI-assisted delivery: better design review, earlier risk detection, clearer trade-offs and stronger governance, with architectural rigor and human accountability kept intact.

Contents

The problem

The most expensive engineering mistakes are architectural ones found late: hidden coupling between services, a missing idempotency guarantee, an unversioned event contract, a weak rollback plan, missing monitoring. By the time these surface in production they are costly to fix.

Architecture review is supposed to catch them, but in practice it is inconsistent. It depends on who happens to review the pull request, how much time they have, and whether they remember the relevant ADRs and runbooks. The reasoning is rarely written down, so the same risks get missed again.

What it does

It turns architecture review into a repeatable, grounded and auditable step. Give it an issue or a pull request and it produces a decision pack:

ArtifactWhat it is
architecture-impact-report.mdThe full assessment across all concerns
adr-draft.mdA draft architecture decision record
test-plan.mdTest levels and the required test cases
pr-review-comment.mdA concise reviewer comment, draft only
decision-log.jsonAn audit record: inputs, retrieved sources, findings, gate
run-metrics.jsonTokens, cost, latency and which backends ran

For an event-driven change, for example, it raises findings such as a new asynchronous dependency, consumer idempotency, event schema versioning, retry and dead-letter handling, queue lag monitoring, deployment sequencing, rollback complexity, contract tests, an ADR requirement and a human approval requirement. Every finding cites the repository files that informed it.

It never merges, deploys or posts to a real pull request. It advises; a human decides.

How it works

It is an agentic system that uses RAG. Two ideas, both doing real work.

Agentic: a coordinated team of agents. The review is split across six specialised agents, each owning one concern, each with an explicit input and output contract, run by an orchestrator that threads a shared state through them:

  1. Context Retriever, 2. Architecture Impact, 3. CI/CD and Release Risk,
  2. Security and Governance, 5. Testing Strategy, 6. ADR and PR Comment.

Later agents build on earlier findings (the security agent aggregates prior risks to decide the approval gate; the ADR agent synthesises everything into a decision). Agents use tools (a retriever, a diff parser, and a structured LLM call that returns schema-validated JSON), and any high or critical finding forces a human approval gate.

RAG: grounding in your repository. Before the agents reason, the tool retrieves the most relevant context from your repo and docs (README, ADRs, runbooks, CI/CD workflows, Terraform, Kubernetes manifests) and injects it into the prompts. The review reflects your repository and your runbooks, not the model's generic training, and the sources are named so a human can verify them.

flowchart TB
    subgraph IN[Inputs]
        A1[GitHub issue file]
        A2[PR diff file]
        A3[Live PR via gh, read only]
    end

    subgraph KNOW[Knowledge sources]
        K1[Repository code]
        K2[ADRs]
        K3[Runbooks]
        K4[CI/CD, Terraform, k8s]
    end

    subgraph RET[1. Retrieve - the R in RAG]
        R1[Load and chunk files]
        R2[Index: TF-IDF or vector + FAISS]
        R3[Rank top context chunks]
    end

    subgraph GEN[2. Augment and 3. Generate - multi-agent reasoning]
        G1[Context Retriever Agent]
        G2[Architecture Impact Agent]
        G3[CI/CD and Release Risk Agent]
        G4[Security and Governance Agent]
        G5[Testing Strategy Agent]
        G6[ADR and PR Comment Agent]
    end

    OUT[Decision pack in outputs/]
    HUMAN[Human approval gate, named reviewer decides]

    IN --> R3
    KNOW --> R1 --> R2 --> R3
    R3 -->|context injected into prompts| G1
    G1 --> G2 --> G3 --> G4 --> G5 --> G6
    G6 --> OUT --> HUMAN

For detail see docs/architecture.md and docs/agent-design.md.

Quickstart

Requires Python 3.11 or newer. These steps run the full demo in the default mock mode: offline, deterministic, and no API key needed.

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Review the bundled sample issue and see the decision pack in outputs/
ai-sdlc review-issue --repo ./sample-enterprise-app --issue ./examples/issue-event-workflow.md

That is the whole setup. To use a real LLM or switch backends, see Configuration and backends.

The commands

The tool installs one command, ai-sdlc, with these subcommands.

CommandInputUse it for
review-issuea markdown issue or note (--issue)reviewing a proposed change before code exists
review-pra --diff file or a live --prreviewing an actual code change
evalgolden cases YAML (--cases)testing the tool's own output quality
compare-modelsan issue or diffcomparing cost and latency across model tiers

review-issue and review-pr review something and write a decision pack to outputs/. eval tests the reviewer itself against golden cases and prints a pass or fail. --repo is always a local path to the repository whose context should ground the review.

# Review a proposed change described in an issue
ai-sdlc review-issue --repo ./sample-enterprise-app --issue ./examples/issue-event-workflow.md

# Review a code change from a local diff file
ai-sdlc review-pr --repo ./sample-enterprise-app --diff ./examples/pr-event-workflow.diff

# Review a live pull request, fetched read only through the GitHub CLI
ai-sdlc review-pr --repo ./sample-enterprise-app --pr 123 --github-repo owner/name

# Test the reviewer against golden cases, optionally with the LLM-as-judge scorer
ai-sdlc eval --cases ./evals/golden_cases.yaml
ai-sdlc eval --cases ./evals/golden_cases.yaml --judge

# Compare cost and latency for the same review across model tiers
ai-sdlc compare-models --repo ./sample-enterprise-app --issue ./examples/issue-event-workflow.md

A guided walkthrough is in docs/demo-script.md.

Run it on your own repository

The commands above use bundled fixtures so the demo runs with zero setup: sample-enterprise-app/ is a stand in target repo, and examples/ holds a sample issue and PR diff. To review your own code, point --repo at a local checkout (this is the RAG corpus) and give it a real issue, diff or pull request.

# Review a live GitHub pull request, read only (needs gh auth)
ai-sdlc review-pr --repo /path/to/your/checkout --pr 42 --github-repo your-org/your-repo

# Review a local change as a diff
git -C /path/to/your/repo diff main...HEAD > /tmp/change.diff
ai-sdlc review-pr --repo /path/to/your/repo --diff /tmp/change.diff

# Review an issue or design note
ai-sdlc review-issue --repo /path/to/your/repo --issue /tmp/issue.md

The full guide, including self-hosted GitHub Enterprise Server and tuning retrieval for your repository layout, is in docs/usage.md.

Configuration and backends

This is the single place for all options: install extras, the LLM provider and how to use a real model, the orchestrator, and the retriever. Set them through environment variables (copy .env.example to .env). Every option has an offline default, so nothing here is required to run the demo in mock mode.

Install options (pip extras)

The [...] in pip install -e ".[dev]" is not an environment. It is pip extras syntax: it installs the project plus optional dependency groups defined in pyproject.toml. Plain pip install -e . installs only the core dependencies (typer, pyyaml), which already runs the full demo in mock mode.

ExtraInstallsNeeded for
devpytestRunning the test suite
openaiopenai SDKLLM provider openai
anthropicanthropic SDKLLM provider anthropic
langgraphlanggraphOrchestrator langgraph
vectorfaiss-cpu, numpyFAISS backend for the vector retriever

Combine extras with commas, for example pip install -e ".[dev,anthropic,langgraph,vector]".

LLM provider (AI_SDLC_LLM_PROVIDER)

  • mock (default): deterministic local responses, no key, fully offline.
  • openai: set OPENAI_API_KEY and OPENAI_MODEL. Needs the openai extra.
  • anthropic: set ANTHROPIC_API_KEY and ANTHROPIC_MODEL. Needs the anthropic extra.

Using a real LLM, step by step

By default the tool uses mock and never calls an external model. To use a real model (this example uses Anthropic Claude; OpenAI is the same with the OPENAI_ variables), from the project root:

  1. Install the provider SDK:

    pip install -e ".[anthropic]"
    
  2. Create your local env file and edit it (it is git-ignored, so your key is not committed):

    cp .env.example .env
    
    AI_SDLC_LLM_PROVIDER=anthropic
    ANTHROPIC_API_KEY=sk-ant-your-key-here
    ANTHROPIC_MODEL=claude-haiku-4-5-20251001
    
  3. Run any command as usual. The banner shows Provider: anthropic, and outputs/run-metrics.json shows a non-zero estimated_cost_usd.

Today only the Architecture Impact Agent calls the model (through a schema guardrail); the other agents are deterministic. If the key or SDK is missing, the tool stops with a clear message rather than silently falling back.

Orchestrator (AI_SDLC_ORCHESTRATOR)

  • auto (default): use LangGraph when installed, else the sequential loop.
  • sequential: the built-in loop, no dependency.
  • langgraph: force LangGraph. Needs the langgraph extra.

Retriever (AI_SDLC_RETRIEVER)

  • tfidf (default): pure-Python lexical ranking, no dependency.
  • vector: embedding cosine search, with a FAISS backend when installed. The FAISS backend needs the vector extra.

The orchestrator and retriever that actually ran are recorded in outputs/run-metrics.json, for example "retriever": "vector:faiss".

How it runs in a real organization

In production this is not run by hand. It runs as a CI/CD step that acts as an automated architecture reviewer on every pull request, drawing context from your real knowledge sources (in-repo docs plus Confluence, Notion or Backstage).

sequenceDiagram
    autonumber
    actor Dev as Developer
    participant PR as Pull request
    participant CI as CI/CD pipeline
    participant Tool as ai-sdlc reviewer
    participant KB as Knowledge sources
    actor Rev as Human reviewer

    Dev->>PR: Open or update a pull request
    PR->>CI: Trigger the pipeline
    CI->>Tool: Run review-pr on the diff
    Tool->>KB: Retrieve context (repo, ADRs, runbooks, wiki)
    KB-->>Tool: Ranked context chunks
    Tool-->>CI: Decision pack, metrics and risk level
    CI->>PR: Post advisory draft comment and set a status check
    PR->>Rev: Notify reviewer (review requested)
    Rev->>PR: Read the advisory comment and decision pack

    alt Risk is HIGH or CRITICAL
        Note over PR,Rev: AI gate is BLOCKING: a named architecture reviewer must approve
        Rev->>PR: Review the architecture risks, approve or request changes
        Dev->>PR: Merge only after approval and green checks
    else Risk is LOW
        Note over PR,Rev: AI gate is ADVISORY: standard peer review still applies
        Rev->>PR: Normal code review, approve or request changes
        Dev->>PR: Merge after approval and green checks
    end

Human review always happens. The risk level only decides whether the AI adds a mandatory gate: high or critical makes the status check blocking, low leaves it advisory. A GitHub Actions example and the knowledge-source integration patterns are in docs/integration.md.

Evaluation and quality

Output quality is measured, not assumed. Two layers run through the same provider abstraction and stay offline in mock mode:

  • Golden cases assert that generated outputs contain the expected findings. Pass or fail, non-zero exit on failure, so it can gate CI.
  • LLM-as-judge (--judge) scores each report against a rubric (risk coverage, governance, actionability, observability). In mock mode it is a deterministic rubric scorer, so it is reproducible and testable.

See docs/evaluation.md. Cost, latency and model tiering are in docs/cost-latency.md.

Design decisions

The choices behind the build, and the reasoning:

  • Offline-first, mock by default. The whole tool runs with no API key and no network. This keeps the demo free, the tests deterministic, and the evals meaningful. A real provider is one environment variable away.
  • Deterministic baseline plus LLM enrichment. Each agent computes a baseline from explicit analysis. The Architecture agent then asks a model for richer, structured findings, validated against a schema, with the baseline as the fallback. You get model quality when available and reproducibility always.
  • Structured outputs with a guardrail. Real models return malformed JSON sometimes. The tool validates, retries with the specific errors, and falls back rather than crashing. This is the difference between a demo and something you would run in a pipeline.
  • Pluggable backends, no lock-in. Orchestration (sequential or LangGraph) and retrieval (TF-IDF or vector with FAISS) sit behind small interfaces, each with a dependency-free default. Adopt the framework when it earns its place.
  • Human-in-the-loop as a control, not a feature. The tool is read-only by design. High risk forces a named approver. Every run leaves an audit record.
  • Evaluation is first-class. Coverage checks and an LLM-as-judge ship with the project, so quality is observable and regressions are caught.

What it is not

  • It is not a chatbot.
  • It is not an autonomous coding agent.
  • It does not merge, deploy or post comments on real pull requests.

Limitations and roadmap

Honest limitations:

  • The GitHub adapter is read-only by design; there is no write path yet.
  • The default retriever is lexical TF-IDF. The vector option uses a local embedder that captures lexical overlap rather than deep semantics until a real embedder is plugged in.
  • Only the Architecture Impact Agent calls a real model today; the others are deterministic.
  • Findings target event-driven and common delivery risk patterns. Other domains need more signal rules.
  • Cost figures are planning estimates, not billing data.

Already in place: a LangGraph orchestrator, a vector retriever with an optional FAISS backend, and a read-only GitHub PR fetch. Next: optional PR comment posting after human approval, a real semantic embedder behind the existing interface, extending genuine LLM reasoning to the remaining agents, tracing, and more golden cases.

A detailed, prioritised roadmap with trade-offs across token economics, guardrails, evaluation, memory and agent context exchange is in docs/roadmap.md.

Documentation

Contributing and license

Issues and pull requests are welcome. Before opening a PR, run the checks:

python -m pytest
ai-sdlc review-issue --repo ./sample-enterprise-app --issue ./examples/issue-event-workflow.md
ai-sdlc eval --cases ./evals/golden_cases.yaml

Writing conventions and engineering principles are in CLAUDE.md.

Licensed under the MIT License. See LICENSE.

Ver o original no GitHub

Bom para

Relacionados