Hallumark
cognis-digital/hallumark
Auditor de alucinações LLM para RAG — leve, scriptável, pronto para CI
Ver no GitHubAuditor de alucinações e grounding para sistemas RAG que detecta desvios factuais em respostas de LLMs sem infraestrutura pesada. Funciona como MCP server nativo, scriptável e self-hostable, integrando-se direto em pipelines CI/CD e agentes de IA.
Instalação
npx convez-ai cognis-digital/hallumark1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Validar acurácia de respostas em sistemas RAG antes de produção
- Gating automático em CI/CD com falha em achados críticos
- Auditoria contínua de agentes IA que usam retrieval de dados
Veredito

Ferramenta especializada e bem-executada para o problema real de hallucinations em RAG. Implementação madura com suporte a múltiplas linguagens e formatos de saída (table, JSON, SARIF), pronta para integração em workflows existentes.
README
A tradução para o português deste README ainda está na fila.
HALLUMARK
LLM hallucination & grounding auditor for RAG systems
AI Security & Governance — securing LLMs, agents, and the MCP supply chain.
pip install cognis-hallumark
hallumark scan . # → prioritized findings in seconds
Contents
- Why hallumark? · Features · Quick start · Example · Architecture · AI stack · How it compares · Integrations · Install anywhere · Related · Contributing
Why hallumark?
LLM hallucination & grounding auditor for RAG systems — without standing up heavyweight infrastructure.
hallumark is single-purpose, scriptable, and self-hostable: point it at a target, get prioritized results in the format your workflow already speaks (table · JSON · SARIF), gate CI on it, and let agents drive it over MCP.
Features
- ✅ Split Claims
- ✅ Audit Record
- ✅ Audit Records
- ✅ Load Records
- ✅ Parse Records
- ✅ Runs on Linux/macOS/Windows · Docker · devcontainer
- ✅ Ports in Python, JavaScript, Go, and Rust (
ports/)
Quick start
pip install cognis-hallumark
hallumark --version
hallumark scan . # scan current project
hallumark scan . --format json # machine-readable
hallumark scan . --fail-on high # CI gate (non-zero exit)
Example
$ hallumark scan .
[HIGH ] HAL-001 example finding (./src/app.py)
[MEDIUM ] HAL-002 another signal (./config.yaml)
2 findings · risk score 5 · 38ms
Architecture
flowchart LR
A[Input: file / dir / API] --> B[Collectors]
B --> C[Rules / Analyzers]
C --> D[Scorer]
D --> E{Reporters}
E --> F[Table]
E --> G[JSON / SARIF]
E --> H[MCP tool -. drives .-> AI agents]
Use it from any AI stack
hallumark is interoperable with every popular way of using AI:
- MCP server —
hallumark mcp(Claude Desktop, Cursor, Cognis.Studio, uncensored-fleet) - OpenAI-compatible / JSON — pipe
hallumark scan . --format jsoninto any agent or LLM - LangChain · CrewAI · AutoGen · LlamaIndex — wrap the CLI/JSON as a tool in one line
- CI / scripts — exit codes + SARIF for non-AI pipelines
How it compares
| Cognis hallumark | explodinggradients | |
|---|---|---|
| Self-hostable, no account | ✅ | varies |
| Single command, zero config | ✅ | ⚠️ |
| JSON + SARIF for CI | ✅ | varies |
| MCP-native (AI agents) | ✅ | ❌ |
| Polyglot ports (JS/Go/Rust) | ✅ | ❌ |
| Open license | ✅ COCL | varies |
Built in the spirit of explodinggradients/ragas, re-framed the Cognis way. Missing a credit? Open a PR.
Integrations
Pipes into your stack: SARIF for code-scanning, JSON for anything, an MCP server (hallumark mcp) for AI agents, and a webhook forwarder for SIEM/Slack/Jira. See docs/INTEGRATIONS.md.
Install — every way, every platform
pip install "git+https://github.com/cognis-digital/hallumark.git" # pip (works today)
pipx install "git+https://github.com/cognis-digital/hallumark.git" # isolated CLI
uv tool install "git+https://github.com/cognis-digital/hallumark.git" # uv
pip install cognis-hallumark # PyPI (when published)
docker run --rm ghcr.io/cognis-digital/hallumark:latest --help # Docker
brew install cognis-digital/tap/hallumark # Homebrew tap
curl -fsSL https://raw.githubusercontent.com/cognis-digital/hallumark/main/install.sh | sh
| Linux | macOS | Windows | Docker | Cloud |
|---|---|---|---|---|
scripts/setup-linux.sh | scripts/setup-macos.sh | scripts/setup-windows.ps1 | docker run ghcr.io/cognis-digital/hallumark | DEPLOY.md (AWS/Azure/GCP/k8s) |
Related Cognis tools
aegis— AI Agent Permission & Access Auditor — surfaces the lethal trifecta of credentials + injection + reachpromptmirror— Prompt-injection & indirect-injection scanner for any LLM context inputledgermind— Local LLM cost & token forensics proxy with anomaly detectionadversa— LLM red-team harness — OWASP LLM Top 10 + MITRE ATLAS attack packsguardpost— Runtime agent firewall — PII redaction, rate limits, policy enforcementaicard— Auto-generated NIST AI RMF / EU AI Act Annex IV model & system cards
Explore the suite → 🗂️ all 170+ tools · ⭐ awesome-cognis · 🔗 cognis-sources · 🤖 uncensored-fleet · 🧠 hermes
Contributing
PRs, new rules, and demo scenarios are welcome under the collaboration-pull model — see CONTRIBUTING.md and SECURITY.md.
⭐ If
hallumarksaved you time, star it — it genuinely helps others find it.
License
Source-available under the Cognis Open Collaboration License (COCL) v1.0 — free for personal, internal-evaluation, research, and educational use; commercial / production use requires a license (licensing@cognis.digital). See LICENSE.