How to hit the ScrapingBee API

ScrapingBee at https://app.scrapingbee.com/api/v1/ fetches a URL for you through headless Chrome and rotating proxies, and returns the HTML (or extracted data).

Authentication

Copy your key from the ScrapingBee dashboard and pass it as api_key. In the examples it's written {{SCRAPINGBEE_API_KEY}}.

Full reference: https://www.scrapingbee.com/documentation/

Scrape a page

render_js=false skips the browser for static pages (cheaper).

curl "https://app.scrapingbee.com/api/v1/?api_key={{SCRAPINGBEE_API_KEY}}&url=https%3A%2F%2Fexample.com&render_js=false"
Run in PostTaco

Extract data with CSS selectors

curl "https://app.scrapingbee.com/api/v1/?api_key={{SCRAPINGBEE_API_KEY}}&url=https%3A%2F%2Fexample.com&extract_rules=%7B%22title%22%3A%22h1%22%7D"
Run in PostTaco