convez.ai
SkillIntegração & APIsIntermediário
tomaibuilder

Site-to-API

tomaibuilder/site-to-api

Converta websites em APIs com descrição em linguagem natural, até sites com login

Ver no GitHub
InicialPluginScore · de 0 a 100

Transforma qualquer website em API usando linguagem natural. Descreva em inglês simples quais dados você quer extrair de um site, inclusive atrás de login, e receba JSON ou CSV limpo. Funciona com autenticação persistente, stealth browsing e renderização JavaScript completa via Browserless.

Instalação

$npx convez-ai tomaibuilder/site-to-api

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
4
Forks
1
Acompanhando
0
Saúde
42%
Último commit
há 5 meses
Versão
—
Issues abertas
0
Contribuidores
1

Casos de uso

  • Extrair dados estruturados de sites sem API pública
  • Monitorar feeds de sites autenticados (redes sociais, painéis internos)
  • Criar pipelines de dados recorrentes e agendados de fontes web
  • Contornar bots detection com stealth browsing e residential proxies

Veredito

Solução prática para web scraping inteligente que resolve problemas reais (login, JavaScript, bot detection). Requer token Browserless pago para cases autenticados, mas prototipa rapidamente e automatiza bem. Excelente para pipelines de dados onde APIs não existem.

README

A tradução para o português deste README ainda está na fila.

🌐 Site-to-API

Turn any website into an API. Tell Claude, in plain English, what data you want from a site — even one behind a login — and get back clean JSON + CSV. Save it as a recipe and re-run or schedule it, and that website now behaves like an API endpoint you own.

Built as a Claude Code skill. Runs on a real cloud browser via Browserless.


Why this is different

Most scrapers die the moment a site needs a login or fights bots. This one doesn't, because it borrows the three things Browserless does that a plain HTTP request can't:

  • Authenticated profiles — log into LinkedIn / Gmail / Instagram / anything once, and every run after that reuses the session. No re-auth loops, no 2FA hell.
  • Stealth + residential proxies — hides navigator.webdriver, spoofs fingerprints, routes through real residential IPs, auto-solves CAPTCHAs. For the genuinely hard targets.
  • A real browser — full JavaScript rendering, infinite scroll, click-throughs. Not a fragile HTML fetch.

You describe the data. Claude compiles it into a recipe and runs it. You get a feed.

60-second demo

npm install
cp .env.example .env          # paste your Browserless token (Prototyping plan for the Instagram step)
npm run -- run example-hn-frontpage

→ output/example-hn-frontpage/latest.csv — the Hacker News front page as structured data.

Then point it at a logged-in site:

node .claude/skills/site-to-api/scripts/setup-profile.mjs instagram https://www.instagram.com/accounts/login/
node .claude/skills/site-to-api/scripts/run-recipe.mjs example-instagram-ad-funnel

→ a swipe-file of every ad funnel running in your feed (advertiser, hook, destination link).

How it works

plain-English request
        │
        ▼
  Claude (site-to-api skill)  ──►  writes a recipe  ──►  run-recipe.mjs
        │                                                    │
   inspects + fixes selectors  ◄────── JSON/CSV ◄────────────┘
        │
        ▼
  re-run on demand  ·  or  schedule.mjs → recurring data feed

A recipe is a small JSON file: the target URL, the steps to reach the data, the fields to pull, and an optional schedule. See recipes/_schema.md.

Smarter authoring (optional Browserless MCP)

This repo ships a committed .mcp.json for the Browserless MCP server. When it's connected, Claude inspects live pages with browserless_smartscraper to discover selectors instead of guessing them — fewer "zero rows" retries. The token is read from the BROWSERLESS_TOKEN environment variable (the config holds no secret, so it's safe in a public repo). Scheduled runs don't use the MCP; they always run through run-recipe.mjs, so a recipe authored with the MCP still runs unattended.

Setup

  1. Node 18+ and a Browserless account and API token from https://www.browserless.io/?utm_source=git&utm_medium=creator&utm_campaign=tom-crawshaw-site. The free tier covers the public examples (like the Hacker News one). The authenticated examples, including Instagram, use Browserless Authenticated Profiles, which needs the Prototyping plan or above. Use code CRAWSHAW for a free month of Prototyping.
  2. npm install
  3. cp .env.example .env and paste your token
  4. Run the no-auth example to confirm it works (above)

Full walkthrough: QUICK_START.md.

Use it well

Own accounts only, no automated purchases, respect each site's Terms of Service. The shipped Instagram recipe deliberately stops at the landing-page link — it never buys anything. Full guardrails: references/ethics-and-limits.md.


Built by Tom Crawshaw — I teach operators to build AI systems like this. Powered by Browserless. MIT licensed.

Ver o original no GitHub

Bom para

Relacionados