Search results as data, from three engines

One request returns ranked results from Google, Bing and DuckDuckGo in a stable envelope, with absolute ranks and ad flags — then feed the URLs straight into scrape or extract.

One call, one envelope

POST or GET /api/search/v1, with an API token carrying the search scope. Providers you name run concurrently, a URL returned by more than one of them appears once, and every result carries the engine that produced it.

Request
curl -X POST https://api.wayfern.com/api/search/v1 \
  -H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "anti detect browser",
    "engines": ["google", "bing", "duckduckgo"],
    "site": "github.com",
    "limit": 50
  }'
Response (abbreviated)
json
{
  "query": {
    "text": "anti detect browser",
    "engines_requested": ["google", "bing", "duckduckgo"]
  },
  "meta": {
    "engines_failed": [
      { "engine": "duckduckgo", "reason": "captcha_detected" }
    ],
    "version": "2.1"
  },
  "results": [
    {
      "rank": 1,
      "absolute_rank": 1,
      "type": "ad",
      "ad": true,
      "url": "https://advertiser.example.com/browser",
      "title": "Sponsored — Browser Automation",
      "description": "Ad copy as the engine rendered it.",
      "engine": "google"
    },
    {
      "rank": 1,
      "absolute_rank": 2,
      "type": "organic",
      "ad": false,
      "url": "https://github.com/example/anti-detect-browser",
      "title": "example/anti-detect-browser",
      "description": "Fingerprint-aware Chromium automation.",
      "engine": "google"
    }
  ],
  "serp_features": [
    {
      "type": "people_also_search",
      "engine": "google",
      "title": "People also search for",
      "items": [
        {
          "query": "browser fingerprint spoofing",
          "url": "https://www.google.com/search?q=browser+fingerprint+spoofing"
        }
      ]
    }
  ],
  "pagination": { "page": 1, "has_more": true, "next_start": 50 }
}

rank

The position among organic results. Ads are counted on a separate counter, so a sponsored row can never push an organic result down a place — the number means what an SEO report needs it to mean.

absolute_rank

The position among every row, ads included, in the order the engine rendered them. Pair it with ad and type to reconstruct the page as a human saw it.

meta.engines_failed

Per-engine failures are reported, not swallowed. If one provider is blocked while the others answer, you get the results plus a named failure — and meta also carries request_id, requested_at and took_ms.

Typed fields, not a query string you concatenate

Site, filetype, date range, language and region are request fields, validated before the call runs. A malformed date comes back as a 400 instead of a silently unfiltered result set, and each engine's own URL syntax is built for you.

text
string · —
The phrase, up to 512 characters. Optional when site or filetype is set — but one of the three is required, or the call is a 400.
engines
string[] · ["google"]
Any of google, bing, duckduckgo; CSV or repeated values on GET. Duplicates are collapsed before anything runs or is billed.
site
string · —
Restrict to one domain, e.g. github.com. No operator string to assemble and no quoting to get wrong.
filetype
string · —
Restrict to one file type, e.g. pdf.
date
string · —
Range as YYYYMMDD..YYYYMMDD. Anything else is rejected as a 400 before a browser is opened.
lang
string · —
Language hint, e.g. en or en-US.
region
string · —
Region / market hint, e.g. US.
filter
boolean · true
Hide near-duplicate results.
features
boolean · true
Return people-also-search blocks in serp_features, for the engines that expose them.

Offsets in, a stop signal out

Ask for up to 200 results per engine and page with a plain offset. The envelope hands back the next offset and whether another page exists, so the loop condition is a field rather than a heuristic.

limit
request
Results per engine. Integer 1–200, default 50. Wayfern fetches as many provider pages as that takes.
start
request
Offset, integer ≥ 0, default 0. It is carried into the organic rank and into absolute_rank, so page two keeps counting where page one stopped.
pagination.page
response
Which page you are on: start ÷ limit, rounded down, plus one.
pagination.has_more
response
True when any requested engine returned a full page — the signal to fetch again rather than guessing from a result count.
pagination.next_start
response
The start value for the next call: start + limit. Loop until has_more is false.

Two calls, on purpose

Search returns ranked results — titles, descriptions and URLs. It does not open those pages, and Wayfern will not quietly scrape them on your behalf or bill you for pages you did not ask for. When you want the content, make a second call: POST /api/v1/scrape for one page as markdown, or POST /api/v1/extract to hand a list of URLs plus a prompt or a JSON Schema and get structured data back. Both live on the Web Data API, so the token making the second call needs the web scope alongside search.

Call two: the URLs you chose, extracted
curl -X POST https://api.wayfern.com/api/v1/extract \
  -H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://github.com/example/anti-detect-browser",
      "https://github.com/example/fingerprint-suite"
    ],
    "prompt": "project name, stated licence, and the primary language",
    "maxPages": 25
  }'

You choose the URLs

Filter the SERP however you like — drop the ads, keep one domain, take the top ten — and send only that list on. Nothing is fetched that you did not name.

A ceiling on the bill

maxPages caps how many source pages extract will fetch — default 10, maximum 100 — so a long result list cannot turn into an unbounded charge.

Cached pages are not billed twice

Extract charges 1 credit per uncached source page, and every source comes back with its status and whether it was billed. Re-running the same extraction over cached pages is what makes iterating on a prompt cheap.

A blocked engine is a status code, not an empty array

Engines push back — captchas, soft blocks, rate limits. Wayfern names what happened instead of returning zero results and calling it a day. If some providers answered, you get their results with the failures listed in meta.engines_failed. If every requested engine failed and nothing usable came back, the sentinel becomes the HTTP status.

captcha_detected
429
A captcha/soft-block page was served by the engine.
rate_limited
429
The upstream engine rate-limited the request.
blocked
403
A hard block was detected (e.g. a /sorry/ page).
search_timeout
504
The search did not complete in time.
internal_error
500
An unexpected error occurred.

Envelope Errors render as a flat { error, code, request_id, message, reason } body, where reason names the engine that failed, and the same id is repeated in the X-Request-ID header. Running out of credits is its own case: a 402 billing_blocked, before any browser opens.

2 credits per provider result page

Search draws on the same credit balance as browser sessions and the rest of the Web Data API — 2 credits for each provider result page actually fetched. Plans start at $19 a month, and non-expiring credit packs are the small way in.

Per provider page, not per call

A request costs 2 credits for every provider result page it fetches. Ask one engine for 10 results and that is one page; ask three engines and each one bills its own pages.

How a limit becomes pages

Page sizes differ by engine: Google returns up to 100 results a page, while Bing and DuckDuckGo page in tens. A limit of 50 is therefore one page on Google and several on the other two. Pages served from the SERP cache are counted the same way.

Free calls stay free

GET /api/search/engines costs nothing, so a client can discover the provider list without spending. Over MCP, list_search_engines is likewise free.

Discover the providers for nothing

The engine list is served from the same registry the search path resolves against, so a provider cannot appear here without being callable.

Engines
curl
curl https://api.wayfern.com/api/search/engines \
  -H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# {"engines":[{"id":"google","label":"Google"},
#             {"id":"bing","label":"Bing"},
#             {"id":"duckduckgo","label":"DuckDuckGo"}]}

Same search over MCP

An agent connected to the Wayfern MCP server gets search_web, which runs the same search code path as the HTTP route: the same three engines, the same 1–200 limit and start offset, and the same 2-credit rate per provider page. The inputs are renamed for agents — query, language, filter_duplicates and include_related — and query is required, where the HTTP route also accepts a site- or filetype-only search. Related queries come back in serp_features when include_related is on.

Ready to scale?

Mint a token with the search scope, send one request, and read ranked results from three engines out of a single envelope.