Turn any set of pages into typed JSON

Give a JSON Schema or a sentence of English, point at URLs or a whole site, and pay only for the pages that were actually fetched.

POST /api/v1/extract
curl -X POST https://api.wayfern.com/api/v1/extract \
  -H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/pricing", "https://example.com/docs/*"],
    "prompt": "every pricing tier with its monthly cost and included seats",
    "schema": {
      "type": "object",
      "properties": {
        "tiers": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": { "type": "string" },
              "monthlyUsd": { "type": "number" },
              "seats": { "type": "integer" }
            },
            "required": ["name", "monthlyUsd"]
          }
        }
      },
      "required": ["tiers"]
    },
    "maxPages": 10
  }'

Authenticate with a wf_ API token carrying the web scope, or with a dashboard session. Every response hands back what the extraction charged in its credits field.

One URL, a list, or a whole site

urls is an array, and the shape of that array is what changes between extracting one page and extracting a whole site.

One URL

"urls": ["https://example.com/pricing"]

Point at a single page when you already know where the data lives. One uncached fetch, 1 credit.

A list of URLs

"urls": ["https://example.com/pricing",
         "https://example.com/enterprise"]

Send a list and the model merges across the pages, preferring the most specific source where they disagree. maxPages still decides how many are fetched, so raise it above its default of 10 for a longer list.

A whole site

"urls": ["https://example.com/docs/*"]

A trailing /* hands the base URL to map, which discovers URLs across that host — sitemaps first, page links second. Discovery runs a map call and is billed as one.

Describe the result either way

A request needs a schema, a prompt, or both — one of the two is required, and sending neither is a 400 rather than a guess. Whichever you send, the extractor reports only what is literally on the page and returns null for anything the sources do not state.

Schema — you own the shape
body.json
{
  "urls": ["https://example.com/pricing"],
  "schema": {
    "type": "object",
    "properties": {
      "tiers": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "monthlyUsd": { "type": "number" }
          },
          "required": ["name", "monthlyUsd"]
        }
      }
    },
    "required": ["tiers"]
  }
}

The extracted object conforms to the schema you sent, so tiers[0].monthlyUsd is a number your types already know about.

Sentence — the model picks the keys
body.json
{
  "urls": ["https://example.com/pricing"],
  "prompt": "every pricing tier with its monthly cost and included seats"
}

With no schema, the values come back under a single data object keyed as your instruction implies — handy for exploring a page before you commit to a shape. Send both and the sentence steers what to look for while the schema fixes the shape.

Cached source pages are not billed

The published rate is 1 credit per uncached source page. Getting a schema right takes a few attempts, and a second attempt over pages still in cache does not cost a browser — so it does not cost a credit. You do not have to take that on faith: every result names each source, what happened to it, and whether it was billed.

Response
200 OK
{
  "success": true,
  "data": {
    "data": {
      "tiers": [
        { "name": "Starter", "monthlyUsd": 0, "seats": 1 },
        { "name": "Growth", "monthlyUsd": 49, "seats": 5 }
      ]
    },
    "sources": [
      { "url": "https://example.com/pricing", "status": "fetched", "billed": true },
      { "url": "https://example.com/docs/limits", "status": "cached", "billed": false },
      {
        "url": "https://example.com/docs/legacy",
        "status": "failed",
        "billed": false,
        "error": "not_found"
      }
    ]
  },
  "credits": 1
}

What the sources array tells you

  • Cache hits come back as status: "cached", billed: false — they cost nothing.
  • A source that fails to load is status: "failed", billed: false, with the reason attached.
  • maxAge defaults to an hour of cache reuse; set it to 0 to force a live fetch and pay for it.
  • The cache is keyed by URL plus the options that change the captured bytes, so a hit needs the same proxy mode and country.

Three sources, one uncached fetch, one credit — the payload above adds up to its own credits field.

maxPages caps the fetching and the invoice

Pointing at /docs/* on a site you have never crawled is exactly the moment a bill can surprise you. It cannot here: maxPages is a hard ceiling on the source pages fetched, and therefore on what the extraction itself charges. A wildcard's map discovery is billed on top, and it is bounded by the same number.

  • maxPages accepts 1 to 100 and defaults to 10.
  • Wildcard expansion, de-duplication and fetching all happen under that ceiling — two overlapping paths never bill the same page twice.
  • Credits are reserved for the worst case (every page uncached) before any page is fetched, then only the pages actually fetched are charged.
  • Run out of credits and the platform hard-stops rather than quietly overspending.

Structured JSON as a scrape add-on

When a single URL is all you need and you also want the markdown, skip /extract and ask /scrape for the json format with a jsonOptions.schema or .prompt. The object lands on data.json next to everything else the page returned.

POST /api/v1/scrape
curl
curl -X POST https://api.wayfern.com/api/v1/scrape \
  -H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/pricing",
    "formats": ["markdown", "json"],
    "jsonOptions": {
      "prompt": "the plan names and their monthly prices"
    }
  }'

How it bills

1 credit for the scraped page plus the +4 credit output add-on — 5 credits in total, charged once however many of screenshot, PDF and JSON you ask for on that page.

One difference worth knowing: a /scrape cache hit is still a billed page, because it is still a page you received. Only extraction prices cache hits at zero.

A worked example

You ask for 40 URLs. 12 of them were fetched recently enough to still be in cache. Here is the invoice.

Source pages requested
40
maxPages: 40, so nothing beyond this is fetched or charged.
Served from cache
12
status: "cached", billed: false — already captured within maxAge.
Actually fetched
28
status: "fetched", billed: true — a real browser had to load these.
Charged
28 credits
1 credit per uncached source page, returned in the response's credits field.

Note Failed sources are never billed either. If one of those URLs had been a /* path, the discovery behind it runs as a map call — 2 credits per started block of 10 returned URLs — which is debited separately and is not included in the extract response's credits field.

At 1 credit per uncached source page, the Developer plan's 1,520 monthly credits cover 1,520 of them — and the same balance also pays for browser sessions, crawls, screenshots and search. See the full rate card.

Ready to scale?

Point the extract endpoint at your first URL, iterate on the schema for free, and keep the ceiling where you put it.