Back to catalogue

Web Extraction

Fetch a public URL, respect robots rules, normalize the page, and return structured metadata, text, markdown, links, and JSON-LD in one paid request.

mainnettestnet Public http/https pages onlyMetadata, text, markdown, links, and JSON-LD extractionRobots-aware fetching with bounded redirects and body size limitsMainnet and testnet route families
Endpoints

2 endpoints available

Click any endpoint to see details, example requests, and try it live with your Freighter wallet.

POST /scrape/extract
mainnet $0.03 USD

Fetch and normalize one public HTML page into structured extraction output.

Example Request JSON
request.json

Edit this JSON before trying the endpoint. The paid retry uses the exact same payload.

CLI Example
curl 
curl -X POST "https://xlm402.com/scrape/extract" -H "Content-Type: application/json" -d @- <<'JSON'
{
  "url": "https://example.com/article",
  "format": "markdown",
  "include_links": true,
  "include_metadata": true,
  "include_json_ld": true,
  "max_chars": 50000
}
JSON
Body Schema
  • url: absolute http|https URL
  • format?: text | markdown
  • include_links?: boolean
  • include_metadata?: boolean
  • include_json_ld?: boolean
  • max_chars?: integer 1000..100000
POST /testnet/scrape/extract
testnet $0.03 USD

Fetch and normalize one public HTML page into structured extraction output.

Example Request JSON
request.json

Edit this JSON before trying the endpoint. The paid retry uses the exact same payload.

CLI Example
curl 
curl -X POST "https://xlm402.com/scrape/extract" -H "Content-Type: application/json" -d @- <<'JSON'
{
  "url": "https://example.com/article",
  "format": "markdown",
  "include_links": true,
  "include_metadata": true,
  "include_json_ld": true,
  "max_chars": 50000
}
JSON
Body Schema
  • url: absolute http|https URL
  • format?: text | markdown
  • include_links?: boolean
  • include_metadata?: boolean
  • include_json_ld?: boolean
  • max_chars?: integer 1000..100000