ScrapingBee fetches a URL and hands back the page. Wayfern does that too — and also runs crawl jobs and hands you a live browser. Here is where each shape fits.
Most people evaluating alternatives do not actually have a ScrapingBee problem — they have a workload that outgrew the fetch-a-URL shape. It is worth knowing which one you have.
Stay on ScrapingBee if the job is "give me this URL as content". Its per-request cost is a published table — 1 credit for a rotating proxy without JavaScript rendering, 5 with rendering (the default), 10 and 25 for premium proxies, 75 for a stealth proxy with rendering — so you can model a scrape budget before you sign up. Auto-Mode goes further: it tries configurations from cheapest to most expensive, charges only for the one that succeeded, obeys a max_cost ceiling, and charges nothing at all when every configuration fails. Feature gating is generous too — JavaScript rendering, rotating and premium proxies, geotargeting, screenshots, extraction rules and the dedicated scraping APIs are all checked on the $49 Freelance tier, and the higher plans buy volume, concurrency and support rather than unlocking capabilities. You can try it with 1,000 free API credits and no credit card, which is more than Wayfern offers: Wayfern has no free grant, and its self-serve ladder starts at Developer at $19/month (1,520 credits, 20 concurrent sessions).
Look at Wayfern when the unit of work stops being one URL. Three things force that move: you need to walk a site rather than name every page yourself; you need results pushed to your pipeline instead of returned in the HTTP response; or you need to stay logged in, click through a flow and read the DOM between steps. ScrapingBee’s API reference documents no crawl or spider endpoint and no webhook, callback or scheduling mechanism — every documented example returns the result in the response body — so that orchestration lives in your code. Wayfern exposes a bounded crawl job with signed webhook delivery, and a CDP WebSocket you attach Playwright or Puppeteer to. If none of those three sentences describes your work, switching buys you nothing.
The single-URL case maps almost one to one. The mapping below only uses parameters documented on both sides.
ScrapingBee takes its options as query parameters on a GET; Wayfern takes them as a JSON body on a POST, with the same bearer-token header you would use anywhere else. The one behavioural difference to plan for: render_js is a switch on their side and an assumption on ours, so a workload that leaned on render_js=false for cheap fetches loses that saving.
The call below bills 1 credit per page for the page plus the +4 credit output add-on for the json format, and the response envelope reports exactly what it charged in its `credits` field, so you can reconcile per call rather than at the end of the month.
# render_js=true (their default) → always on; every call renders in Chromium
# render_js=false → no equivalent
# stealth_proxy=true → "proxy": "residential"
# country_code=us → "country": "us"
# return_page_markdown → "formats": ["markdown"]
# return_page_source → "formats": ["rawHtml"]
# screenshot / screenshot_full_page→ "formats": ["screenshot"]
# extract_rules / ai_query → "formats": ["json"] + jsonOptions
# js_scenario wait / wait_for → "waitFor" (ms) / "waitForSelector"
# timeout (max 140000) → "timeout" (ms), clamped to 120000
curl -X POST https://api.wayfern.com/api/v1/scrape \
-H "Authorization: Bearer $WF_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com/pricing",
"formats": ["markdown", "json"],
"proxy": "auto",
"country": "us",
"waitForSelector": "main article",
"jsonOptions": { "prompt": "the plan names and their monthly prices" }
}'Rendering, proxies and geotargeting are table stakes on both sides. These are the rows where the answers genuinely differ.
| Capability | Wayfern | Them |
|---|---|---|
| Unit of work | A REST call per page, or a crawl job with an id you poll. A CDP socket is a third unit: one browser held open across many navigations. | One GET per supplied URL; the result comes back in the HTTP response. |
| Multi-page crawling | POST /api/v1/crawl: page limit, link depth, include/exclude path regexes, sitemap seeding, robots.txt obeyed by default, per-site delay and in-job concurrency. | No crawl or spider endpoint in the API reference — each call fetches the single URL you supply, so link discovery is left to the caller. |
| Result delivery | Poll GET /api/v1/crawl/:id, or receive HTTPS webhooks for started/page/completed/failed events, each carrying an X-Wayfern-Signature: sha256=… HMAC over the delivery timestamp and body. | No webhook, callback or scheduling mechanism in the API reference; every documented example returns the result in the HTTP response. |
| Interactive control | Attach Playwright or Puppeteer over CDP and drive the page yourself; read the DOM between steps and branch on what you find. | js_scenario, a JSON instruction list (click, wait, wait_for, fill, evaluate, infinite_scroll and others) submitted with the request. |
| Skipping the browser | Not offered. Every Web Data call renders in a pooled Chromium; blockAssets only skips image, media and font downloads. | render_js=false fetches without a headless browser — the cheapest row on the credit table. |
| Exit selection | proxy: auto | datacenter | residential plus an ISO-3166 country. auto starts on a datacenter exit and retries residential only when a navigation is classified as blocked. | Rotating, premium and stealth proxies, each chosen per request, with country_code (ISO 3166-1) geotargeting and separate supported-country lists per proxy type. |
| Identity held across a run | One browser context for the whole CDP session, with an optional sticky residential session id on the gateway URL. | Not documented in the researched pages. (not published) |
| Structured extraction | The json format with a JSON Schema or a plain-English prompt (+4 credits per page), or POST /api/v1/extract at 1 credit per source page, where re-running over already-cached pages is free. | extract_rules with CSS or XPath selectors, or AI extraction via ai_query / ai_extract_rules with an optional ai_selector, priced at 5 credits on top of the base request cost. |
| Per-page time ceiling | A caller-supplied timeout, clamped to 120,000 ms. | timeout defaults to 140000 ms and accepts 1000–140000 ms, with a stated 0.5 second margin of error — a longer ceiling than Wayfern allows. |
Both sell credits. They meter completely different things, so a per-credit comparison is noise.
ScrapingBee meters the request and prices it by how hard the request was: 1 credit for a rotating proxy without rendering, 5 with rendering, 10 and 25 for premium proxies without and with rendering, 75 for a stealth proxy with rendering, and 5 more on top when ai_query or ai_extract_rules is used. Plans are published — Freelance $49, Startup $99, Business $249, Business + $599, all excluding VAT, plus a Custom tier priced on request — and each buys a monthly credit allowance and a concurrency cap. The allowance is valid for one month and does not roll over. What this meter punishes is difficulty: a site that forces you to stealth-with-rendering costs 75x a plain fetch, which is exactly why Auto-Mode and its max_cost ceiling exist.
Wayfern meters the endpoint. A scraped page is 1 credit per page, a crawled page 1 credit per successful page, an extraction 1 credit per source page, a capture 5 credits per capture, a web search 2 credits per search; asking a page for a screenshot, PDF or JSON output adds +4 credits and a summary adds +2. The rate does not change when a site is hard — but the exit does: residential egress is metered per byte at 320 credits per GB on top of the endpoint rate, while datacenter egress is free. A held browser session bills 8 credits per open-page hour with a 60-second minimum per page. What this meter punishes is bandwidth and wall-clock time: a heavy residential run can cost more than the endpoint rate suggests.
Self-serve plans start at Developer at $19/month (1,520 credits, 20 concurrent sessions). Plan credits, like ScrapingBee’s, expire at the end of the period; one difference is that separately purchased top-up packs do not expire and carry over. If your workload is a fixed set of URLs on cooperative sites, ScrapingBee’s meter is easier to forecast. If it is a crawl whose page count you cannot predict, a flat per-page rate is.
A request/response scraping API discards its browser after every call; a held CDP session keeps one browser, one cookie jar and one exit identity for the whole run.
ScrapingBee is a request/response API. You issue a GET, it renders the page with a headless browser when render_js is on (it is on by default), and it returns HTML, Markdown, structured JSON or a screenshot. Interaction happens through js_scenario, a JSON instruction list — click, wait, wait_for, wait_for_and_click, scroll_x, scroll_y, fill, evaluate, infinite_scroll — that you submit with the request and cannot inspect while it runs. That is a clean model, and for a stateless fetch it is the right one.
Wayfern runs both models against the same pool. The Web Data endpoints are request/response like ScrapingBee’s — POST a URL, get Markdown, HTML, links, metadata, a screenshot, a PDF or schema-shaped JSON back. The gateway is not: connecting Playwright to wss://browser.wayfern.com/ws?token=… opens a real Chromium you drive statement by statement, assert against, and keep for as long as the login or the multi-step flow needs. Nothing is submitted blind, because you are holding the page.
The cost of that flexibility is honest to state: every Wayfern Web Data call loads the page in a pooled Chromium. There is no cheaper non-browser mode to drop into the way ScrapingBee’s render_js=false gives you a 1-credit fetch for pages that do not need rendering.
Not answerable without your workload, and anyone who answers it flatly is guessing. ScrapingBee charges per request on a difficulty curve (1, 5, 10, 25 or 75 credits depending on proxy and rendering, plus 5 for AI extraction); Wayfern charges a flat rate per endpoint — 1 credit per page for a scrape, 1 credit per successful page for a crawled page — and meters residential bandwidth separately at 320 credits per GB. Easy pages through a rotating proxy are cheap on their meter; large crawls of ordinary pages are cheap on ours. Price both against a real sample before you decide.
Yes. POST /api/v1/scrape is the same shape as a ScrapingBee GET — one URL in, page content out, at 1 credit per page, with markdown, html, rawHtml, links, screenshot, pdf, json and summary as the available formats. Nothing about Wayfern requires you to adopt crawl jobs or CDP sessions.
No, and this is a genuine point for ScrapingBee: signing up there gives you 1,000 free API credits with no credit card. Wayfern grants no free credits — the self-serve ladder starts at Developer at $19/month (1,520 credits, 20 concurrent sessions).
Two things, depending on how much control you need. On the Web Data endpoints there is no declarative instruction list at all — only waitFor, waitForSelector, custom headers, a mobile viewport and include/exclude selectors. For anything requiring clicks, form fills or branching on page state, you attach Playwright or Puppeteer to the CDP gateway and write the steps as ordinary code, which also means you can read the page between steps instead of submitting a script blind.
A crawl returns a job id immediately. From there you either poll GET /api/v1/crawl/:id for paginated results and GET /api/v1/crawl/:id/errors for the pages that failed, or register an HTTPS webhook and receive started, page, completed and failed events as they happen — each delivery signed with an HMAC over its timestamp and body, so a leaked URL is not enough to forge results into your pipeline. ScrapingBee documents no webhook or callback mechanism, so this is code you would otherwise write and operate yourself.
Browser sessions, the Web Data API, search and captcha solving all draw on one credit balance. Start on any of them.