convez.ai
MCP ServerAutomaçãoIntermediário
creandoaldia

Wafle-Scraper

creandoaldia/wafle-scraper

Web scraping seguro e ético para agentes de IA com isolamento total

Ver no GitHub
InicialPluginScore · de 0 a 100

Servidor MCP universal para web scraping seguro com automação de navegador. Permite que agentes de IA extraiam dados da web em modo incógnito, sem cookies ou dados privados, com rate limiting e bloqueio de IPs locais. Implementa detecção de CAPTCHA com solução interativa manual.

Instalação

$npx convez-ai creandoaldia/wafle-scraper

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
v0.1.0
Issues abertas
0
Contribuidores
1

Casos de uso

  • Extração de dados públicos de websites para análise
  • Automação de navegação com interação (cliques, digitação, scroll)
  • Monitoramento de conteúdo web com respeito a rate limits
  • Coleta de dados de APIs públicas (ex: Reddit) de forma segura

Veredito

Solução robusta e bem pensada para scraping ético. O foco em segurança (incógnito obrigatório, bloqueio de IPs privados, rate limiting) e a recusa de resolver CAPTCHAs automaticamente são escolhas corretas. Recomendado para quem precisa extrair dados públicos com responsabilidade.

README

A tradução para o português deste README ainda está na fila.

██╗    ██╗  █████╗  ███████╗██╗     ███████╗
██║    ██║ ██╔══██╗ ██╔════╝██║     ██╔════╝
██║ █╗ ██║ ███████║ █████╗  ██║     █████╗
██║███╗██║ ██╔══██║ ██╔══╝  ██║     ██╔══╝
╚███╔███╔╝ ██║  ██║ ██║     ███████╗███████║
 ╚══╝╚══╝  ╚═╝  ╚═╝ ╚═╝     ╚══════╝╚══════╝

Wafle-Scraper — Universal MCP Server for Safe Web Scraping

Wafle-Scraper is an MCP server that lets AI agents (OpenCode, Claude, Cursor) extract data from the web safely and responsibly.

Safety First

RuleEnforcement
No private dataEvery browser session is incognito — no cookies, no localStorage, no saved passwords
Only what you askThe scraper never mines extra data beyond your explicit request
No localhostInternal/private IPs are blocked by default
Rate limitedMinimum 2 seconds between requests — never floods servers
CAPTCHA = human onlyNo automated CAPTCHA solving. If one appears, you solve it interactively
User-Agent rotationEach request looks like a real browser

Quick Start

pip install wafle-scraper
playwright install chromium
wafle-scraper

Configuration

OpenCode / Claude Desktop / Cursor

{
  "mcpServers": {
    "wafle-scraper": {
      "command": "wafle-scraper",
      "description": "Web scraping & browser automation — incognito, audited, safe"
    }
  }
}

CLI Options

wafle-scraper                    # MCP stdio mode (default for agents)
wafle-scraper --http --port 8000 # HTTP SSE mode
wafle-scraper --version          # Show version

MCP Tools

ToolDescriptionSafety
scrape_urlExtract text from a static URL (requests + BeautifulSoup)✅ Read-only, no JS
scrape_browserNavigate a page in isolated incognito browser and extract text✅ Incognito, no cookies
scrape_redditFetch public posts from a subreddit via official API✅ API, no scraping
extract_emailsFind email addresses on a public page✅ Only what you ask
browser_interactClick, type, scroll, extract, screenshot in live browser✅ You control the actions

Browser Backend (Playwright)

  • Incognito always: storage_state=None, fresh context per session
  • No permissions: No camera, mic, location access
  • Anti-detection: Rotating UA, viewport, locale, timezone
  • Natural delays: Human-like timing between actions
  • Gradual scroll: Loads lazy content naturally

CAPTCHA Handling

Wafle-Scraper does NOT solve CAPTCHAs automatically. When a CAPTCHA is detected:

  1. The scraper pauses
  2. Prompts you to open the URL in your browser
  3. You solve the CAPTCHA manually
  4. Type done and the scraper continues

This is the only ethical and reliable approach without paid services.

Security

  • Blocked: localhost, 127.0.0.1, private IPs, file://, chrome://
  • Rate limiting (configurable, default 2s min interval)
  • Scope enforcement — only processes what you explicitly request
  • User-Agent rotation
  • Browser isolation — Playwright contexts are fully sandboxed

Requirements

  • Python 3.10+
  • Playwright with Chromium installed (playwright install chromium)
  • Windows, macOS, Linux

Dependencies

Standalone — no other MCP servers required.

Known Issues

IssueDetailStatus
CDP health check unstable_cdp_health_check() creates a throwaway context just to eval 1+1. Masks real CDP issues instead of diagnosing them.🟡 Medium
CAPTCHA handler is interactive onlyCAPTCHA solver relies on stdin — doesn't work in MCP server mode (no stdin available in stdio transport).🔴 High
_safe_goto only 2 retriesNo exponential backoff, no proxy fallback. If 2 attempts fail, the whole scrape fails.🟡 Medium
Error strings instead of exceptionsbrowser_navigate, browser_click, etc. return "Error: ..." strings instead of raising. Consumers must parse strings.🟡 Medium
Single browser contextOnly 1 context at a time. Concurrent scrape_browser calls cause race conditions.🔴 High
Reddit HTML fallback is silentIf JSON API fails, falls back to old.reddit.com without logging the fallback path.🟢 Low

Installation from Source

git clone https://github.com/creandoaldia/wafle-scraper.git
cd wafle-scraper
pip install -e .
playwright install chromium

License

MIT

Why "Wafle-Scraper"?

Part of the WAFLE ecosystem (Web AI Framework for Language Ecosystems). Wafle-Scraper gives WAFLE agents the ability to read the live web — safely, transparently, and under your control.

Ver o original no GitHub

Bom para

Relacionados