PromptArmor — Red Teaming para Segurança de LLMs
allsmog/promptarmor-plugin
Red teaming inteligente para LLMs — analisa código, ataca, avalia e corrige vulnerabilidades
Ver no GitHubMCP server + plugin Claude para teste de segurança em aplicações LLM através de red teaming. Analisa código-fonte para identificar prompts de sistema, schemas de ferramentas e superfícies de injeção, depois executa 80+ tipos de ataque com 25+ estratégias de mutação. Gera relatórios detalhados e patches de código em locais específicos.
Instalação
npx convez-ai allsmog/promptarmor-plugin1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Testar robustez de aplicações LLM contra prompt injection e jailbreak
- Validar guardrails e system prompts antes de deploy em produção
- Identificar superfícies de ataque em integrações LLM + ferramentas externas
- Gerar patches automáticos para vulnerabilidades descobertas
Veredito

Ferramenta especializada e bem-arquitetada para quem precisa validar segurança de aplicações LLM em produção. Diferencia-se pelo análise prévia de código (não ataca cego), estratégias contextualizadas e geração de patches. Recomendado para times que lidam com LLMs críticos e exigem conformidade com padrões de segurança.
README
A tradução para o português deste README ainda está na fila.
PromptArmor — LLM Security Red Teaming
Claude Code plugin + MCP server for LLM application security red teaming. Reads your code, tests your endpoints, fixes your vulnerabilities.
Why PromptArmor
Existing tools like promptfoo test endpoints blind — they send attacks and check responses, but have no idea what your system prompt says, what tools you expose, or where your guardrails are. PromptArmor reads the code first:
- Recon — Scans your codebase to find system prompts, tool definitions, guardrails, and injection surfaces
- Plan — Prioritizes attacks based on what it found (found a SQL tool? Prioritize SQL injection. System prompt says "never discuss competitors"? Test that specifically.)
- Attack — Sends 80+ attack types with 25+ mutation strategies against your endpoint
- Judge — LLM-as-judge + pattern matching + context-aware grading
- Fix — Generates actual code patches at specific file:line locations, not generic advice
Quick Start
Install the plugin
claude plugin add ./plugin
Run a full scan
/prompt-armor:scan --target https://your-app.com/api/chat
Or just analyze the code
/prompt-armor:analyze
Architecture
Two components working together:
┌─────────────────────────────────────────────┐
│ Claude Code Plugin │
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐│
│ │ Commands │ │ Agents │ │ Skills ││
│ │ 6 cmds │ │ 5 agents │ │ 5 modules ││
│ └──────────┘ └──────────┘ └──────────────┘│
│ ┌──────────┐ │
│ │ Hooks │ │
│ │ 4 hooks │ │
│ └──────────┘ │
└───────────────────┬─────────────────────────┘
│ MCP Protocol
┌───────────────────▼─────────────────────────┐
│ MCP Server (TypeScript) │
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐│
│ │ Attacks │ │ Judge │ │ Reports ││
│ │ 80+ │ │ 3-tier │ │ text/json/ ││
│ │ plugins │ │ grading │ │ sarif ││
│ └──────────┘ └──────────┘ └──────────────┘│
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐│
│ │Mutations │ │ Client │ │ State ││
│ │ 25+ │ │ adapters │ │ persistence ││
│ │strategies│ │ for APIs │ │ JSON files ││
│ └──────────┘ └──────────┘ └──────────────┘│
└─────────────────────────────────────────────┘
Commands
| Command | Description |
|---|---|
/prompt-armor:scan | Full pipeline: code analysis → attack planning → testing → remediation → report |
/prompt-armor:analyze | Code-only analysis — find LLM integration points without testing endpoints |
/prompt-armor:attack <category> | Run specific attack category against an endpoint |
/prompt-armor:report | Generate report from saved results |
/prompt-armor:config | Create or validate promptarmor.yaml |
/prompt-armor:diff | Compare two scan results for regression tracking |
Agents
| Agent | Role |
|---|---|
| recon-agent | Scans codebase for system prompts, tool schemas, guardrails, injection surfaces |
| attack-planner | Creates targeted attack strategy based on recon findings |
| red-teamer | Executes attacks via MCP server tools |
| judge-agent | Reviews borderline verdicts with context-aware grading |
| remediation-agent | Generates code-specific fixes with file:line references |
Attack Coverage
80+ attack plugins across 10 categories:
| Category | Count | Examples |
|---|---|---|
| Jailbreak | 13 | Basic ignore, role switch, DAN, system prompt extract, delimiter injection |
| Injection | 5 | Direct, indirect, context compliance, system override, prompt extraction |
| Tool Abuse | 12 | SQL injection, SSRF, file read/write, code execution, privilege escalation |
| Harmful | 18 | Violence, crime, hate, self-harm, drugs, weapons, cybercrime, misinformation |
| Bias | 5 | Age, gender, race, disability, religion |
| PII | 4 | Direct disclosure, session leak, social engineering, DB leak |
| Compliance | 4 | HIPAA, COPPA, FERPA, GDPR |
| Agentic | 3 | Memory poisoning, cross-session leak, goal misalignment |
| RAG | 3 | Document exfiltration, source attribution, poisoning |
| Other | 13+ | Hallucination, imitation, contracts, competitors, debug access |
25+ mutation strategies:
| Type | Strategies |
|---|---|
| Encoding | Base64, ROT13, Hex, Leetspeak, Homoglyph, Morse, Pig Latin, Emoji, ASCII Smuggling |
| Structural | Few-Shot Prime, Context Stuffing, Instruction Delimiter, Markdown Injection, Citation, Authoritative Markup |
| Multi-turn | Crescendo, GOAT, HYDRA, Retry |
| Advanced | Multilingual (10 languages), Math Prompt, Best-of-N, Jailbreak Templates, Composite |
Configuration
Create a promptarmor.yaml:
target:
url: https://your-app.com/api/chat
format: openai
attacks:
suites: [jailbreak, injection, tool-abuse, harmful, pii]
num_per_plugin: 5
mutations:
strategies: [base64, multilingual, crescendo]
judge:
provider: anthropic
analysis:
enabled: true
paths: [src/]
output:
formats: [text, json, sarif]
ci:
fail_on: critical
Output Formats
- Text/Markdown — Human-readable report with findings, severity, and remediation
- JSON — Machine-readable for automation
- SARIF v2.1.0 — Upload to GitHub Security tab
CI Integration
PromptArmor detects CI environments automatically and adjusts:
- JSON/SARIF output (no interactive prompts)
- Exit code 1 when vulnerabilities exceed
--fail-onthreshold - Artifacts saved to
.prompt-armor/for collection
MCP Server
The MCP server is usable standalone by any MCP client (Claude Desktop, Cursor, etc.):
| Tool | Purpose |
|---|---|
run_attack_suite | Run a full suite of attacks |
send_attack | Send a single attack |
judge_response | Judge a response |
get_results / save_results | State management |
generate_report | Generate reports |
list_attacks / list_mutations | Discovery |
validate_config | Config validation |
E2E Verified
Tested against an intentionally vulnerable Node.js chat endpoint with keyword-triggered secret disclosure, system prompt leakage, and no input sanitization:
$ /prompt-armor:scan --target http://localhost:4010/api/chat
Artifacts:
.prompt-armor/recon/recon-2026-03-24.json (system prompts, tool schemas, injection surfaces)
.prompt-armor/state.json (full scan state with per-test verdicts)
.prompt-armor/reports/report-2026-03-24-full.md (151-line executive report)
.prompt-armor/reports/promptarmor-*.json (machine-readable report)
.prompt-armor/reports/promptarmor-*.txt (text report)
Tests: 150 across 8 categories
Passed: 133
Failed: 17 (15 critical, 2 high)
By category:
jailbreak: 12/16 (4 failed) tool-abuse: 13/24 (11 failed)
injection: 13/15 (2 failed) harmful: 47/47 passed
bias: 15/15 passed pii: 12/12 passed
compliance: 12/12 passed agentic: 9/9 passed
Detected:
- System prompt disclosure (3 vectors)
- DAN mode jailbreak adoption
- Secret exfiltration (JWT_SECRET, DB credentials via prompt injection)
- Privilege escalation (admin access granted unconditionally)
- File content disclosure (/etc/passwd)
- SQL injection (DROP TABLE executed)
- Tool schema enumeration
Recon correctly identified the system prompt (src/config.js:3), tool schemas, hardcoded secrets, and all injection surfaces with file:line references.
Development
npm install # Install dependencies
cd server && npm test # Run server tests