Wafle-Scraper
creandoaldia/wafle-scraper
Web scraping seguro e ético para agentes de IA com isolamento total
Ver no GitHubServidor MCP universal para web scraping seguro com automação de navegador. Permite que agentes de IA extraiam dados da web em modo incógnito, sem cookies ou dados privados, com rate limiting e bloqueio de IPs locais. Implementa detecção de CAPTCHA com solução interativa manual.
Instalação
npx convez-ai creandoaldia/wafle-scraper1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Extração de dados públicos de websites para análise
- Automação de navegação com interação (cliques, digitação, scroll)
- Monitoramento de conteúdo web com respeito a rate limits
- Coleta de dados de APIs públicas (ex: Reddit) de forma segura
Veredito

Solução robusta e bem pensada para scraping ético. O foco em segurança (incógnito obrigatório, bloqueio de IPs privados, rate limiting) e a recusa de resolver CAPTCHAs automaticamente são escolhas corretas. Recomendado para quem precisa extrair dados públicos com responsabilidade.
README
A tradução para o português deste README ainda está na fila.
██╗ ██╗ █████╗ ███████╗██╗ ███████╗
██║ ██║ ██╔══██╗ ██╔════╝██║ ██╔════╝
██║ █╗ ██║ ███████║ █████╗ ██║ █████╗
██║███╗██║ ██╔══██║ ██╔══╝ ██║ ██╔══╝
╚███╔███╔╝ ██║ ██║ ██║ ███████╗███████║
╚══╝╚══╝ ╚═╝ ╚═╝ ╚═╝ ╚══════╝╚══════╝
Wafle-Scraper — Universal MCP Server for Safe Web Scraping
Wafle-Scraper is an MCP server that lets AI agents (OpenCode, Claude, Cursor) extract data from the web safely and responsibly.
Safety First
| Rule | Enforcement |
|---|---|
| No private data | Every browser session is incognito — no cookies, no localStorage, no saved passwords |
| Only what you ask | The scraper never mines extra data beyond your explicit request |
| No localhost | Internal/private IPs are blocked by default |
| Rate limited | Minimum 2 seconds between requests — never floods servers |
| CAPTCHA = human only | No automated CAPTCHA solving. If one appears, you solve it interactively |
| User-Agent rotation | Each request looks like a real browser |
Quick Start
pip install wafle-scraper
playwright install chromium
wafle-scraper
Configuration
OpenCode / Claude Desktop / Cursor
{
"mcpServers": {
"wafle-scraper": {
"command": "wafle-scraper",
"description": "Web scraping & browser automation — incognito, audited, safe"
}
}
}
CLI Options
wafle-scraper # MCP stdio mode (default for agents)
wafle-scraper --http --port 8000 # HTTP SSE mode
wafle-scraper --version # Show version
MCP Tools
| Tool | Description | Safety |
|---|---|---|
scrape_url | Extract text from a static URL (requests + BeautifulSoup) | ✅ Read-only, no JS |
scrape_browser | Navigate a page in isolated incognito browser and extract text | ✅ Incognito, no cookies |
scrape_reddit | Fetch public posts from a subreddit via official API | ✅ API, no scraping |
extract_emails | Find email addresses on a public page | ✅ Only what you ask |
browser_interact | Click, type, scroll, extract, screenshot in live browser | ✅ You control the actions |
Browser Backend (Playwright)
- Incognito always:
storage_state=None, fresh context per session - No permissions: No camera, mic, location access
- Anti-detection: Rotating UA, viewport, locale, timezone
- Natural delays: Human-like timing between actions
- Gradual scroll: Loads lazy content naturally
CAPTCHA Handling
Wafle-Scraper does NOT solve CAPTCHAs automatically. When a CAPTCHA is detected:
- The scraper pauses
- Prompts you to open the URL in your browser
- You solve the CAPTCHA manually
- Type
doneand the scraper continues
This is the only ethical and reliable approach without paid services.
Security
- Blocked:
localhost,127.0.0.1, private IPs,file://,chrome:// - Rate limiting (configurable, default 2s min interval)
- Scope enforcement — only processes what you explicitly request
- User-Agent rotation
- Browser isolation — Playwright contexts are fully sandboxed
Requirements
- Python 3.10+
- Playwright with Chromium installed (
playwright install chromium) - Windows, macOS, Linux
Dependencies
Standalone — no other MCP servers required.
Known Issues
| Issue | Detail | Status |
|---|---|---|
| CDP health check unstable | _cdp_health_check() creates a throwaway context just to eval 1+1. Masks real CDP issues instead of diagnosing them. | 🟡 Medium |
| CAPTCHA handler is interactive only | CAPTCHA solver relies on stdin — doesn't work in MCP server mode (no stdin available in stdio transport). | 🔴 High |
_safe_goto only 2 retries | No exponential backoff, no proxy fallback. If 2 attempts fail, the whole scrape fails. | 🟡 Medium |
| Error strings instead of exceptions | browser_navigate, browser_click, etc. return "Error: ..." strings instead of raising. Consumers must parse strings. | 🟡 Medium |
| Single browser context | Only 1 context at a time. Concurrent scrape_browser calls cause race conditions. | 🔴 High |
| Reddit HTML fallback is silent | If JSON API fails, falls back to old.reddit.com without logging the fallback path. | 🟢 Low |
Installation from Source
git clone https://github.com/creandoaldia/wafle-scraper.git
cd wafle-scraper
pip install -e .
playwright install chromium
License
MIT
Why "Wafle-Scraper"?
Part of the WAFLE ecosystem (Web AI Framework for Language Ecosystems). Wafle-Scraper gives WAFLE agents the ability to read the live web — safely, transparently, and under your control.