// Use case · Extraction
For pipelines that need fields, not HTML: send a JSON Schema or one sentence, point at URLs or a whole site, and pay only for the pages actually fetched.
credit per uncached page
1
pages per request, max
100
credits for a cache hit
0
{ "success": true, "data": { "data": { "tiers": [ { "name": "Starter", "monthlyUsd": 0 }, { "name": "Growth", "monthlyUsd": 49 } ] }, "sources": [ { "status": "fetched", "billed": true }, { "status": "cached", "billed": false } ] }, "credits": 1}// Why
The problem
What Wayfern does
prompt, or both. Fields the pages do not state come back null.maxPages (1 to 100, default 10) caps both the fetching and the bill.// How it works
Authenticate with a wf_ token that carries the web scope. The extraction runs synchronously and returns the data, the sources and the charge together.
Step 1 of 4
One URL, a list, or a trailing /* that discovers a site's URLs through map first.
"urls": ["https://example.com/docs/*"]
Step 2 of 4
A JSON Schema fixes the shape; a sentence lets the model pick the keys. One of the two is required.
"schema" or "prompt"
Step 3 of 4
Uncached pages load in a real browser. Pages captured within maxAge (one hour by default) are reused.
"maxPages": 10
Step 4 of 4
Each source says fetched, cached or failed and whether it was billed. credits is the total.
"credits": 1
// Code
A schema and a prompt together: the prompt steers what to look for, the schema fixes the shape.
curl -X POST https://api.wayfern.com/api/v1/extract \
-H "Authorization: Bearer wf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://example.com/pricing", "https://example.com/docs/*"],
"prompt": "every pricing tier with its monthly cost and included seats",
"schema": {
"type": "object",
"properties": {
"tiers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"monthlyUsd": { "type": "number" },
"seats": { "type": "integer" }
},
"required": ["name", "monthlyUsd"]
}
}
},
"required": ["tiers"]
},
"maxPages": 10
}'// Targets
urls is an array. Its shape is what changes between extracting one page and extracting a site.
Point at one page when you know where the data lives. One uncached fetch, 1 credit.
"urls": ["https://example.com/pricing"]
The model merges across the pages and prefers the most specific source where they disagree. maxPages still decides how many are fetched: raise it above 10 for a longer list.
"urls": ["…/pricing", "…/enterprise"]
A trailing /* hands the base URL to map, which discovers URLs across that host: sitemaps first, page links second. Discovery is billed as a map call.
"urls": ["https://example.com/docs/*"]
// Describe the result
With a schema, the object conforms to it, so tiers[0].monthlyUsd is a number your types already know. With only a sentence, the values come back under one data object keyed as your instruction implies: useful for exploring a page before you commit to a shape.
{
"urls": ["https://example.com/pricing"],
"schema": {
"type": "object",
"properties": {
"tiers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"monthlyUsd": { "type": "number" }
},
"required": ["name", "monthlyUsd"]
}
}
},
"required": ["tiers"]
}
}A source served from cache comes back as status: "cached", billed: false.
A source that fails to load is status: "failed", billed: false, with the reason attached.
maxAge defaults to one hour of cache reuse; set it to 0 to force a live fetch and pay for it.
The cache is keyed by URL plus the options that change the captured page, so a hit needs the same proxy mode and country.
// What it costs
Re-running an extraction over pages still in cache costs nothing, which is what makes iterating on a schema cheap. At this rate the Developer plan’s 1,520 monthly credits cover 1,520 uncached pages.
maxPages: 40. Cached and failed sources are not billed.
| Item | Credits |
|---|---|
| 28 uncached source pages | 28 |
| 12 pages served from cache | 0 |
Total
28credits
about $0.35
at $0.0125 / credit
About 54 runs a month on Developer ($19 for 1,520 credits).
/docs/* with maxPages: 50. The map call behind the wildcard is debited separately.
| Item | Credits |
|---|---|
| map: 80 URLs returned (2 per 10) | 16 |
| 50 uncached source pages | 50 |
Total
66credits
about $0.83
at $0.0125 / credit
About 23 runs a month on Developer ($19 for 1,520 credits).
The same balance pays for browser sessions, crawls, screenshots and search. See the full rate card.
// FAQ
No. Send a schema, a prompt, or both; sending neither is a 400. With no schema, values come back under one data object keyed as your instruction implies. Send both and the prompt steers what to look for while the schema fixes the shape.
That field comes back null. The extractor reports only what is on the page, rather than a guess.
No. maxPages is a hard ceiling on source pages fetched; wildcard expansion, de-duplication and fetching all happen under it, so two overlapping paths never bill a page twice. Credits are reserved for the worst case before any fetch, then only pages actually fetched are charged. The map call behind a /* costs 2 credits per started block of 10 returned URLs and is not included in the extract response's credits.
Not necessarily. Map discovers URLs across the host, so the result can include pages outside that path. Pass an explicit list of URLs when the exact scope matters.
Yes. Ask /scrape for the json format with jsonOptions.schema or jsonOptions.prompt; the object lands on data.json beside the markdown. That bills 1 credit for the page plus the +4 output add-on. A scrape cache hit is still billed; only extract prices cache hits at zero.
The request stops rather than overspending. Plan credits are spent first, then pack credits; at zero the platform hard-stops unless auto top-up is on.
// Keep reading
// Next step
Point the extract endpoint at one URL, iterate on the schema over cached pages for free, and keep the page ceiling where you set it.