Firecrawl alternatives: when a content API is the wrong shape

Where Firecrawl is still the right answer

Two panels, both true. If the left one describes your workload, stop reading and keep what you have.

The wall a content API hits is state, not difficulty

A request/response scrape discards everything between calls; a held CDP session keeps one browser, one cookie jar and one exit identity for the whole run.

Firecrawl narrows this gap deliberately and it is worth being precise about how. Its scrape endpoint takes an `actions` array — write, press, click, wait, screenshot — so a page can be poked before it is captured. Its Interact endpoint scrapes a page and then interacts with it using AI prompts or code, billed at 2 credits per browser minute, and it documents a Browser Sandbox offering managed browser sessions for interactive workflows. For a login form and a button, that is enough.

It stops being enough when the sequence is long, branching, or written in code you already own. Wayfern inverts the relationship: instead of describing the interaction inside a request body, you connect your own Playwright or Puppeteer client to a browser we hold open, and every line of automation you have already written keeps working. The gateway authenticates a link token on the WebSocket URL, enforces the link concurrency cap, and hands back a live CDP endpoint.

The cost model follows the shape. An open page bills 8 credits per hour, prorated per second with a 60-second minimum per page — so a session is priced by how long you hold it, not by how many pages you managed to squeeze through it.

Their docs, our code, row by row

Firecrawl cells come from the sources listed at the bottom of this page. Wayfern cells describe endpoints and options that exist in the running service. Where their docs do not answer a row, it says so rather than guessing.

Pages fetched versus browser-time and bytes

Both sell credits. They are not measuring the same thing, so compare workloads, not rate cards.

Firecrawl meters work done per page and publishes it plainly: scrape, crawl and map are 1 credit per page, Monitor is 1 credit per page per check, Search is 2 credits per 10 results, and Interact is 2 credits per browser minute. A PDF costs a credit per page of the PDF. An auto-mode scrape that needed the enhanced proxy costs 5. That model is easy to forecast, which is a real advantage when you are budgeting a crawl of a known size.

Wayfern meters the same way for the same work — 1 credit per scraped page, 1 per successfully crawled page, 2 per 10 discovered URLs, 1 per uncached source page for extract, 5 per screenshot, 2 per provider result page for search — with add-ons priced rather than bundled: +4 once per page for a screenshot, PDF or JSON output, +2 for an LLM summary. Re-running an extraction over pages already cached is free, so iterating on a schema does not cost anything.

The part with no Firecrawl equivalent is the second meter: 8 credits per open-page hour (60-second minimum), and residential bandwidth at 320 credits per GB debited live from the container's own byte counters as the session spends them. Datacenter egress is free. Both meters draw on one balance, so a run that scrapes 900 pages and then holds a browser open for twenty minutes is one number, not two invoices.

Plans start at $19/month for 1,520 credits and 20 concurrent sessions. Credit packs are one-time and roll over; the monthly grant does not. Firecrawl's published tiers run from a free 1,000 credits a month up to Scale, though note their pricing page renders the annually-billed rate by default, so the monthly-billing number is not the one on screen.

What each one does when a site says no

Wayfern tries a datacenter exit first and retries through a residential exit only when the navigation is classified as blocked — every retried byte is metered.

Firecrawl exposes the choice as three proxy modes. basic is fast and usually works. enhanced is slower but more reliable on complex sites while maintaining privacy. auto retries with enhanced when basic fails, and a scrape rescued that way bills 5 credits rather than the regular cost; a scrape that succeeded on basic bills the regular cost.

Wayfern splits the same decision across two axes instead of one. proxy is auto, datacenter or residential: auto goes out through a datacenter exit and retries through a residential one only when the failure is a bot wall — a timeout or a DNS failure is surfaced immediately rather than burned through an expensive retry. country picks the exit geography. The retry is not a flat surcharge: it is the residential bytes it actually moved, at the standard per-GB rate.

That difference matters in both directions. A page that fails on the cheap exit and succeeds on the expensive one has a predictable, capped cost on Firecrawl and a usage-shaped cost on Wayfern. If your blocked pages are small, per-byte is cheaper; if they are heavy and you dislike variance, a flat 5 credits is easier to defend in a budget meeting. On the browser gateway the same bytes are watched by a monitor that samples the container every few seconds and terminates a session that can no longer pay for its own bandwidth, which is the safety valve that makes per-byte tolerable.

Porting a scrape call, and the call that has no port

The stateless half moves almost mechanically. formats, onlyMainContent, includeTags, excludeTags, waitFor and maxAge keep their names and meanings; a Firecrawl location.country becomes the top-level country field; a proxy mode of auto maps to proxy: "auto", and reaching for enhanced on every request maps to proxy: "residential". The response envelope is { success, data, credits } — the credits the call actually debited come back with the data, so you can meter per call without a separate usage query.

Two things do not port. There is no document parser on this side: if you rely on Firecrawl reading a PDF or DOCX at a URL, keep it for that. And nothing in a Wayfern request body replaces an actions array — the interaction moves out of the request and into your own Playwright script against the CDP gateway, which is more code and a great deal more control.

curl -X POST https://api.wayfern.com/api/v1/scrape \
  -H "Authorization: Bearer $WF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/pricing",
    "formats": ["markdown", "links"],
    "onlyMainContent": true,
    "excludeTags": ["nav", "footer"],
    "waitFor": 1200,
    "maxAge": 3600000,
    "proxy": "auto",
    "country": "us"
  }'

# { "success": true, "data": { "markdown": "…", "metadata": {…} }, "credits": 1 }

# The half that has no request-body equivalent:
#   const browser = await chromium.connectOverCDP(
#     "wss://browser.wayfern.com/ws?token=<LINK_TOKEN>&country=US&session=run42"
#   );

Questions teams ask before they move