The best web scraping tools, sorted by what you are actually buying

Six tools, one axis: how much browser you get. Every competitor claim on this page was read from that vendor’s own published documentation before it shipped.

First decide whether you are buying pages or sessions

Almost every bad purchase in this category is the same mistake: buying browser capacity for a job a single parsed request would have done, or buying a parsed request for a job that only exists inside a session. Settle this before you compare anyone.

Buy parsed pages when the target is public, the answer is already in the HTML, and one call should hand back clean markdown or typed JSON. This is the cheaper and much larger half of the market. Firecrawl charges 1 credit per page for scrape, crawl and map; ScrapingBee charges 1 credit for a rotating-proxy fetch and 5 with JavaScript rendering. If that describes your job, a cloud browser is capacity you will pay for and never drive.

Buy a held browser when the data sits behind a login, a multi-step form, a cart or an infinite scroll — anywhere the second request only makes sense because of what the first one did. That run needs one cookie jar, one fingerprint and one exit IP from start to finish, and it needs you at the controls. Wayfern, Browserbase and Browserless all sell this shape; Bright Data sells it as its Browser API alongside its proxy networks.

Six tools, from most page-shaped to most session-shaped

Numbered along one axis — how much browser you get, from a single parsed response to a browser you drive yourself. That is a reading order, not a score. Wayfern sits at the session end because that is what Wayfern is, not because this is our page.

  1. 01

    ScrapingBee

    A single GET endpoint that fetches one URL, with the proxy tier, JavaScript rendering, geotargeting, screenshots and extraction rules all selected per request.

    Best for Teams who want the cost of a scrape to be arithmetic. The credit table is published per configuration (1, 5, 10, 25 or 75), and Auto-Mode escalates from cheapest to most expensive, charges only for the tier that worked, and charges 0 credits when every tier fails.

  2. 02

    Firecrawl

    An AGPL-3.0 API that turns pages and documents into LLM-ready markdown or schema-driven JSON, with crawl, map, search, batch and agent endpoints around it.

    Best for RAG and agent pipelines that want a page-shaped API with unusually transparent operational limits — per-plan, per-endpoint rate limits, concurrency and queue depths are all published — plus the option to run the engine in-house.

  3. 03

    Bright Data

    A proxy platform — datacenter, ISP, residential and mobile — with the Web Unlocker, a hosted Browser API and a Crawl API layered on top.

    Best for Jobs whose hard problem is the exit IP. Residential targeting reaches US state, city, ZIP code and ASN, and the unblocking APIs bill per successful request by default rather than per attempt.

  4. 04

    Browserless

    Hosted browsers reachable three ways: a BaaS v2 WebSocket your existing Puppeteer or Playwright code connects to, stateless REST endpoints, and BrowserQL, a GraphQL API over HTTPS.

    Best for Anyone who needs an engine other than Chromium. It documents Chromium, Chrome, a stealth build, Firefox, WebKit and Edge — with the honest caveat that the last three are Playwright-only and cannot use stealth mode.

  5. 05

    Browserbase

    Session-first cloud browsers driven by Playwright, Puppeteer or Selenium, created through a REST API whose browserSettings cover stealth, captcha solving, ad blocking, recording and allowed domains.

    Best for Teams who want session state and session evidence handled for them: browserSettings expose recordSession and logSession, contexts persist cookies, localStorage and IndexedDB between sessions, and four browser regions are selectable.

  6. 06

    Wayfern

    An anti-detect browser you drive over CDP, plus a per-page Web Data API, three-engine search and captcha solving, all metered from one credit balance.

    Best for Runs that need one identity end to end: choose the residential exit down to country, region, city, ISP and a sticky session id on the WebSocket URL, then hold that browser for the whole job.

What a held session gives you that a response cannot

A request-shaped API returns one document and throws the browser away. A held CDP session keeps the cookie jar, the DOM and the exit IP for the whole run.

REQUEST / RESPONSE APIyour codetarget sitestate discarded between callsHELD CDP SESSIONyour codebrowsercookies · DOMtarget siteone identity for the whole run
A request-shaped API returns one document and throws the browser away. A held CDP session keeps the cookie jar, the DOM and the exit IP for the whole run.

The split is not about quality, it is about where state lives. A page-shaped call hands back what the vendor’s browser saw and then discards that browser, so step two of a logged-in, paginated flow starts from nothing. A session-shaped call hands you the browser and bills you for how long you hold it.

Every vendor selling sessions has had to add a state carrier back, and each one is capped somewhere. Browserbase contexts persist cookies, localStorage, IndexedDB, session storage, service workers and browser preferences, but explicitly exclude the HTTP cache, so each session re-fetches page assets from the network. Browserless authenticated profiles capture cookies, localStorage and IndexedDB, but cap at 2 MB of state and 50 origins per profile and are removed after 30 days unused. Bright Data bounds the session itself: five minutes idle times out, sixty minutes is the maximum, and a session’s navigation is confined to one domain.

Wayfern’s browser gateway has no create-session call at all. Opening the CDP WebSocket at wss://browser.wayfern.com/ws with a link token is what creates the session, and it lives exactly as long as your socket does. Open-page time bills at 8 credits an hour with a 60-second minimum per page, and the residential exit you selected with ?country=, ?region=, ?city=, ?isp= and ?session= stays fixed for the run — the pool will only reuse a container that was spawned with the same upstream proxy.

How each one meters, and what the meter hides

Not what it costs — what the unit is. Mismatched units are why two quotes in this category are almost never comparable.

Three units are in play. ScrapingBee and Firecrawl meter requests: ScrapingBee charges 1, 5, 10, 25 or 75 credits depending on proxy tier and whether JavaScript rendered, and Firecrawl charges 1 credit per page for scrape, crawl and map plus 2 credits per browser minute for Interact. Browserbase meters browser hours, with proxy gigabytes billed on top. Bright Data meters successful requests on its unblocking APIs. Wayfern meters both shapes out of one balance: 1 credit per scraped page on the Web Data API, 8 credits per open-page hour on the browser gateway.

Success-based billing carries a footnote worth reading. Bright Data’s Web Unlocker bills only successful requests by default, but the docs state that enabling custom headers or cookies switches billing to 100% of requests, successful and failed alike. ScrapingBee ships the cleanest version of the idea: Auto-Mode tries configurations cheapest-first, charges only for the one that succeeded, honours a max_cost ceiling, and costs 0 credits when every configuration fails.

Residential bandwidth is where quotes stop being comparable at all. Wayfern meters it per byte from a per-container sidecar sampled every five seconds, debits 320 credits per gigabyte as it is consumed, and terminates a session that can no longer pay for it; datacenter egress is free. Browserbase bills proxy gigabytes separately from browser hours. Bright Data’s own per-gigabyte residential rate was not readable from any first-party page while this was written, so no figure for it appears here — the third-party numbers in circulation contradict each other.

Last, check what the price gates rather than meters. ScrapingBee checks JavaScript rendering, rotating and premium proxies, geotargeting, screenshots and extraction rules on its $49 entry plan, so higher tiers buy volume, concurrency and support rather than features. Browserbase gates the other way: its docs state Verified — the tier with real browser fingerprints — is offered on the Scale plan, its top tier, so the strongest anti-detection sits behind the largest commitment. Wayfern’s three plans differ only in monthly credits, concurrent sessions and seats; every endpoint and the same browser build is on all of them, plan credits expire at the end of the period, and credit packs bought on top do not expire and roll over.

Where Wayfern actually sits in this list

The Wayfern column is checked against the running service. The other column names the specific tool the claim belongs to, read from that vendor’s own published documentation. Three of these rows go against us; leaving them out would make the rest worth less.

CapabilityWayfernThem
Unit of work you buyBoth. 1 credit per scraped page on the Web Data API, 8 credits per open-page hour on the CDP gateway, drawn from one credit balance.Firecrawl and ScrapingBee sell parsed pages; Browserbase sells browser hours; Bright Data sells successful requests plus proxy bandwidth.
Browser enginesChromium-family only — Wayfern’s own anti-detect build, driven with Playwright or Puppeteer over CDP.Browserless documents six engines including Firefox, WebKit and Edge, though those three are Playwright-only and cannot use stealth. Browserbase’s Create Session API exposes no engine parameter at all.
Exit-location targetingcountry, region, city, isp and a sticky session id, set as query parameters on the /ws CDP URL; a country is the minimum to request a residential exit.Bright Data residential reaches country, US state, city, US ZIP and ASN (datacenter and ISP are country-only). Browserbase targets country, state and city across a stated 201 countries. Browserless targets country and city with sticky sessions.
Published residential bandwidth rate320 credits per GB, metered per byte and debited live as the session consumes it; datacenter egress is free.Bright Data’s per-GB residential rate (not published)
Strongest anti-detection tier by planThe same browser build and the same residential exits on every plan. Plans differ only in monthly credits, concurrent sessions and seats.Browserbase states Verified is offered on the Scale plan, its top tier. ScrapingBee is the opposite case: rotating and premium proxies, geotargeting and JavaScript rendering are all checked on its $49 entry tier.
Crawl orchestrationPOST /api/v1/crawl returns a job id, bills 1 credit per successfully crawled page, delivers signed webhooks as pages land, and charges nothing for polling.Firecrawl, Browserless (/crawl) and Bright Data (Crawl API) all ship crawl endpoints. ScrapingBee’s API reference documents none — link discovery is left to the caller.
Concurrency on a published self-serve plan20 concurrent sessions on Developer, 100 on Business, 400 on Scale (open pages, not requests).ScrapingBee 50/100/200/400 concurrent requests. Browserbase 3/25/100/250+ concurrent browsers, with overflow returning 429 rather than queueing. Firecrawl 2/5/50/100/150+ concurrent browsers. Units differ between vendors.
Official SDKsNone published. The gateway is standard CDP, so any Playwright or Puppeteer client works, and the REST API is plain JSON over HTTP.Firecrawl publishes SDKs for Python, Node.js, Go, Rust, Java and Elixir plus a CLI. Browserbase publishes Node.js and Python.
Self-hostingNot a published plan — Wayfern’s three tiers are hosted subscriptions.Firecrawl is AGPL-3.0 with a first-party self-hosting guide, though its own SELF_HOST.md says self-hosted instances do not get Fire-engine. Browserless publishes Docker and private-deployment options. Browserbase documents no self-hosted or on-premise deployment.

Questions buyers ask before they pick one

What is the real difference between a scraping API and a cloud browser?

Where the state lives. A scraping API returns the document its browser saw and then discards that browser, so nothing carries over to the next call. A cloud browser hands you the browser and bills you for how long you hold it, so a login, a form and a paginated result set are one continuous run. Most public-page work is cheaper as the former; anything that only exists inside a session is only possible as the latter.

Do I actually need residential proxies?

Usually not on the first attempt. On Wayfern datacenter egress is free and residential is metered per byte at 320 credits per GB, so the cheap path is to try datacenter and escalate only when you are blocked. ScrapingBee has productised that escalation: Auto-Mode tries configurations cheapest-first, charges only for the one that worked, and charges 0 credits if none do. Worth knowing before you plan around it: Bright Data’s residential network is not self-serve — personal-email accounts are ineligible, access is granted only to registered companies, and approval goes through human-reviewed KYC returning within 48 hours.

Which of these can I run on my own infrastructure?

Firecrawl and Browserless. Firecrawl is AGPL-3.0 with a first-party self-hosting guide, but its own SELF_HOST.md states that self-hosted instances "do not have access to Fire-engine, which includes advanced features for handling IP blocks, robot detection mechanisms, and more" — so the hardest part of scraping is exactly what the self-hosted build lacks. Browserless publishes Docker and private-deployment options. Browserbase documents no self-hosted or on-premise deployment, and self-hosting is not one of Wayfern’s published plans either.

Where do anti-detect browsers like Multilogin fit in?

They solve a neighbouring problem — many persistent, individually fingerprinted profiles for account work, rather than throughput. Multilogin exposes roughly fifteen fingerprint vectors per profile, each with explicit Masked, Custom and Real modes, but its automation only works through a remote WebDriver ("Using a local WebDriver will break automation") and its API rate limit tops out at 100 requests per minute shared across a whole workspace. Bright Data takes the same view from the other side: it ships no anti-detect browser of its own and lists 31 third-party ones as proxy integration targets.

What does a 100,000-page scrape cost on Wayfern?

Scrape is 1 credit per page, so 100,000 pages is 100,000 credits before add-ons; asking for a screenshot, PDF or JSON output adds 4 credits per page and an LLM summary adds 2. Those credits come either from a plan grant, which expires at the end of the billing period, or from credit packs, which do not expire and roll over — the 20,000-credit pack is $200. Residential bandwidth, if the job needs it, is metered separately per byte.

When is Wayfern the wrong tool here?

When the job is a large volume of public pages that render without a session. Paying for held browser time to do work one parsed request would have done is the most common way to overspend in this category, and the page-shaped tools on this list are built for exactly that. Wayfern is also Chromium-family only: if you need Firefox or WebKit, Browserless documents both — through Playwright, and without stealth mode.

Try it on your own targets

Browser sessions, the Web Data API, search and captcha solving all draw on one credit balance. Start on any of them.