Async APIs
Submit long-running page, SERP, and batch scraping jobs, then poll for the finished result.
Some endpoints submit work asynchronously and return a job identifier instead of waiting for the final payload. Use async requests for long-running pages, SERP jobs, and async commerce workloads.
Async flow map
| Feature | Endpoint | Returned id | Polling result |
|---|---|---|---|
| WebScrapingAPI async | GET /v2?async=1&url=... on api.webscrapingapi.com | response_id | Final scraped body or JSON wrapper |
| SERP async | GET /v2?engine=google_async&q=... on serpapi.webscrapingapi.com | response_id | Final SERP payload |
| Async commerce | POST /v1?engine=<engine>_async on ecom.webscrapingapi.com | snapshot_id | Final JSON records |
Web Unblocker async
WebScrapingAPI /v2 accepts these async fields:
| Parameter | Behavior |
|---|---|
url | Target page to scrape. |
method | Target request method. |
body | Target request body for body-capable requests. |
country | 2-letter country code for geotargeting. |
render_js=1 | Enables JavaScript rendering. |
Use country for async geotargeting. state, city, and asn are not supported for async /v2 submission.
SERP async
The google_async engine submits a SERP request and returns a response_id.
| Parameter | Behavior |
|---|---|
| SERP engine fields | Added to the target search request. |
device=mobile | Requests mobile SERP results. |
device=tablet | Requests tablet SERP results. |
response=json | Requests structured JSON where supported. |
country | Applies country-level geotargeting when present. |
Async batch engines
These engines submit a batch input array and return a snapshot_id:
| Engine | Request mode | Input |
|---|---|---|
bestbuy_async | Async batch | Best Buy product URLs or keyword inputs; see Best Buy Async. |
booking_async | Async batch | body array; type selects hotel, hotel pricing, or search inputs. |
chatgpt_async | Async batch | body array of prompt inputs. |
nordstrom_async | Async batch | body array of Nordstrom product URLs. |
target_async | Async batch | Target product or discovery inputs; see Target Async. |
wayfair_async | Async batch | body array of Wayfair product URLs. |
Poll the returned snapshot_id until the result is ready. Completed responses are returned as JSON records.
Batch submit example
curl --request POST \
"https://ecom.webscrapingapi.com/v1?api_key=YOUR_API_KEY&engine=bestbuy_async&type=product" \
--header "Content-Type: application/json" \
--data '[
{
"url": "https://www.bestbuy.com/product/apple-airpods-pro-3-wireless-active-noise-cancelling-earbuds-with-heart-rate-sensing-feature-white/JJGCQLYK5F"
}
]'Response:
{
"snapshot_id": "s_m4x7enmven8djfqak"
}Poll:
curl --get "https://ecom.webscrapingapi.com/v1" \
--data-urlencode "api_key=YOUR_API_KEY" \
--data-urlencode "snapshot_id=s_m4x7enmven8djfqak"Finished results remain available for a limited download window. Poll promptly and persist completed results in your own storage.