convez.ai
SkillDevOps & InfraestruturaIntermediário
HetCreep

CoalMine – Canários de Qualidade para Agentes IA

HetCreep/CoalMine

9 auditorias de saúde de código para agentes IA evitarem regressões

Ver no GitHub
BomPluginScore · de 0 a 100

Skill que fornece 9 canários de qualidade para detectar code rot, alucinações, vulnerabilidades de supply-chain, falhas de resiliência e desvios de contrato API antes de poluírem o codebase. Executa auditorias automáticas e sob demanda com evidência zero-grade-inflation, seguindo ciclo seguro de stash/commit → fix → build/test → auto-revert.

Instalação

$npx convez-ai HetCreep/CoalMine

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
15
Forks
1
Acompanhando
0
Saúde
71%
Último commit
há 1 dia
Versão
v3.28.3
Issues abertas
1
Contribuidores
2

Casos de uso

  • Detectar dead code, resource leaks e race conditions no final da sessão
  • Auditar vulnerabilidades de dependências, licenças e segurança de CI/CD
  • Verificar resiliência (FMEA, rollbacks, idempotência) e performance (N+1 queries)
  • Prevenir hallucinations de IA via source-grounding obrigatório
  • Validar conformidade com padrões de projeto (gold-standard) e drift de contratos API

Veredito

CoalMine é uma ferramenta sólida de quality assurance para agentes IA, com 9 canários bem-definidos e modos de execução flexíveis (always-on, auto+manual, on-demand). Recomendado para equipes que querem guardrails técnicos robustos e automatizados, embora exija integração manual em agentes além de Claude.

README

A tradução para o português deste README ainda está na fila.

🐤 CoalMine

9 Quality-Safeguard Canaries for AI Coding Agents — Detect code rot, weak rules, hallucinations, supply-chain vulnerabilities, brittle architectures, and API contract drift before they pollute your codebase.

version license SKILL.md skills agents

Design Principles · Eval Results · Changelog · Security · Privacy · Releases

Part of TheColliery — Sibling of CoalTipple.


🐤 The 9 Canaries

Skill NameCatchesRun Mode
rot-canaryDead code, bugs, resource leaks, race conditions, silent failures, stale docsAuto + Manual (runs on session end / manual trigger)
gold-standardAudits project completeness against world-class exemplarsOne-time (triggered once, governs the session)
source-groundingPrevents AI hallucinations by forcing cross-source verificationAlways-on (background rule for all chat sessions)
supply-chain-auditAudits dependency vulnerabilities, licenses, phone-home code, and build/CI securityOn-demand (manually run when relevant)
resilience-auditAudits failure path handling (FMEA), rollbacks, retry limits, and idempotencyOn-demand (manually run when relevant)
telemetry-canaryAudits observability, log structures, metrics, and telemetry qualityOn-demand (manually run when relevant)
testability-canaryAudits testing ease, code coupling, mockability, and Dependency Injection (DI)On-demand (manually run when relevant)
scale-canaryAudits performance scaling issues, $O(N^2)$ loops, and duplicate (N+1) database queriesOn-demand (manually run when relevant)
drift-canaryPrevents contract and schema drift (API/database contract inconsistencies)On-demand (manually run when relevant)

Run Mode Details:

  • 📌 Always-on: Runs implicitly in the background to verify facts.
  • 🔄 Auto + Manual: Scans affected files at session end via lifecycle hooks (auto-wired in Claude Code; manual snippets in platform-configs/hooks/ for other agents). Manual trigger via /rot-canary.
  • ⚡ One-time: Governs the session by scanning and filling project-local rules.
  • 🎯 On-demand: Manually run for specific tasks to conserve tokens.

Canaries follow grounding in evidence, zero grade inflation, and report before fixing. Fixes apply through a safe loop: Stash/Commit -> Apply fix -> Run build+tests -> Auto-revert if tests fail.


🔌 Universal Agent Support

SKILL.md is an open standard compatible with all major AI coding agents:

AI AgentTarget Skills FolderInstallation ShortcutChoice Tool Support
Claude Codeplugin cache (recommended) or ~/.claude/skills//plugin install coalmine@coalmine✅ Native: AskUserQuestion
Antigravity.agents/skills/node scripts/install.mjs antigravity✅ Native: built-in question prompt
Cursor.cursor/skills/node scripts/install.mjs cursor✅ Native: built-in ask-question tool
Windsurf.windsurf/skills/node scripts/install.mjs windsurf✅ Native: suggested_responses
GitHub Copilot.github/skills/node scripts/install.mjs copilot✅ Native: askQuestions
Cline.claude/skills/node scripts/install.mjs cline✅ Native: ask_question
Gemini CLI.gemini/skills/node scripts/install.mjs gemini✅ Native: ask_user
Goose.agents/skills/node scripts/install.mjs goose⚠️ Text Fallback: no question tool
Amp.agents/skills/node scripts/install.mjs amp⚠️ Text Fallback: tool not documented
Junie.junie/skills/node scripts/install.mjs junie⚠️ Text Fallback: tool not documented
Codex.agents/skills/node scripts/install.mjs codex✅ Native: request_user_input

Skill paths follow the cross-vendor Agent Skills spec. Cline reads .claude/skills/, Junie reads .junie/skills/, others use .agents/skills/.

What ports where

PartPortable?
The 9 skills (the audits)✅ All targets natively via Agent Skills spec
Interactive choice menus (ask_question)✅ Native question tools on most agents; text fallback on Goose/Amp/Junie
Sub-agent fan-out + tiers✅ Supported if host has sub-agent system; inline fallback
rot-canary auto-cadence✅ Auto-wired on Claude Code; 🔧 manual snippets in platform-configs/hooks/ for other major agents; ⛔ unsupported on Cline/Junie

Manual Fallback: Copy conformed skill body from plugin/skills/<name>/SKILL.md (strip YAML frontmatter) into AGENTS.md / rules file.


🚀 Installation & Verification

Option A — Claude Code Plugin (No clone needed)

/plugin marketplace add HetCreep/CoalMine
/plugin install coalmine@coalmine

🔧 Maintainers: plugin/ is generated output. After edits in skills/, skills/_shared/, hooks/, or .claude-plugin/plugin.json, run node scripts/build-plugin.mjs.

Option A2 — skills.sh (One line)

npx skills add HetCreep/CoalMine

Option B — Universal Installer

1. Clone the Repository

git clone https://github.com/HetCreep/CoalMine.git

2. Run the Installer

Run from your project's root folder (not inside the CoalMine clone):

cd /path/to/your-project
node /path/to/CoalMine/scripts/install.mjs <agent|all|PATH>
  • all auto-detects and installs to all configured agents in the directory.
  • The installer sets up pre-commit/pre-push gates in .git/hooks, writes trigger rules, and generates .coalmine.json config.

3. Verify & Uninstall

  • Verify: node /path/to/CoalMine/scripts/verify.mjs <agent|PATH>
  • Uninstall: node scripts/install.mjs --uninstall <agent|PATH>

🔋 One button: install — the suite drives itself

Installing is the power button. The agent conducts the canaries and asks for consent before running expensive tasks:

WhatWhen it firesYour part
gold-standardOffered once on new projects, and again when a rule's revalidate date passesRun now / Queue / Skip
rot-canaryAuto-scans touched files at session end (QUICK); findings end with a fix menuChoose a fix option
SpecialistsOffered when conversation enters their domain (deps, schemas, async, loops, etc.)Accept / Skip
source-groundingAlways-on background fact verification—

Consent Rule: Nothing expensive runs silently. Revocable via .coalmine.json, ~/.claude/.rot-canary-off, or --uninstall.


🛡️ Work Execution Gate & Haldane Safety

  1. Work Execution Gate: Before starting a task, the agent presents a confirm menu:
    • ทำทันที / Do now — Assess scope, recommend tier, and execute.
    • เก็บเข้าแผนงาน / Add to plan — Queue in task.md.
    • ดูแผนงานทั้งหมด / View full plan — View queued tasks, adjust tiers, and run.
  2. Haldane Safety Protocol (for sub-agents):
    • Active files edited by sub-agents are marked [/] in-flight in task.md to prevent collisions.
    • If conversation shifts to topics affecting in-flight files, the agent warns you first.
  3. Proactive Suggestions: The agent automatically offers canary runs via ask_question when relevant changes (e.g., adding a package) are detected.

⚙️ Configuration (.coalmine.json)

  • General Users (Zero-Config): Automatically generated .coalmine.json pre-configured with safe, token-optimal defaults.
  • Programmers (Overrides): Inline comments document every key. Run the configurator tool or edit manually.

Configuration Schema

KeyTypeDefaultDescription
languageStringautoOverride language detection (en | th | ja | zh | es)
defaultTierStringautoForce execution tier (Light | Standard | Heavy)
autoScanFileCapNumber10Maximum touched files to scan at session end
tripwireMaxFileSizeKbNumber100Size limit in KB for editor conflict checks
enableConductorBooleantrueSet false to disable rules injection on Session Start
disabledCanariesArray[]Canaries to disable (e.g. ["rot-canary"] or ["all"])

Configurator Utility

# Set language and cap limit
node scripts/configure.mjs --language th --file-cap 15

# Disable specific canaries
node scripts/configure.mjs --disable rot-canary,drift-canary

📝 Ultra-Short Summary Format

Canaries report in a lean shape (one-line verdict + severity table of confirmed findings) to save tokens:

| # | path:line | category | severity | finding | evidence |

Severity levels: CRITICAL · HIGH · MEDIUM · LOW. Clean scan outputs a single line.


⚡ Escalation Tiers

TierTriggerOrchestrationToken Cost
LightSmall scope / targeted reviewPrimary agent, quickVery Low 🟢
StandardModerate scope / module reviewMulti-threaded routing, detailedModerate 🟡
HeavyLarge scope / release prepSub-agent fan-out, deep pathsHigh 🔴

📊 Measured detection quality

AV-Comparatives-style eval harness results scored mechanically over 16 fixtures:

CanaryEngineRecallPrecisionDecoy FPsSeverity accuracy
rot-canaryclaude-fable-5 (Baseline)100% (13/13)100%0/413/13
rot-canaryAntigravity (Blind)100% (13/13)100%0/412/13

Measured 2026-06-13 (v3.4.0). Detailed log: RESULTS.md.


🧭 Design Principles

Bound by the 11 principles of the Quantum Computer Spec: maximum performance, zero visible errors, single-brand, minimum power, essential accessories, error correction, determinism, isolation, trustworthiness, and entanglement.


📄 License

MIT License. See LICENSE for details.

Ver o original no GitHub

Bom para

Relacionados