WebScrapingAPI Docs

Async APIs

Submit long-running page, SERP, and batch scraping jobs, then poll for the finished result.

Some endpoints submit work asynchronously and return a job identifier instead of waiting for the final payload. Use async requests for long-running pages, SERP jobs, and async commerce workloads.

Async flow map

FeatureEndpointReturned idPolling result
WebScrapingAPI asyncGET /v2?async=1&url=... on api.webscrapingapi.comresponse_idFinal scraped body or JSON wrapper
SERP asyncGET /v2?engine=google_async&q=... on serpapi.webscrapingapi.comresponse_idFinal SERP payload
Async commercePOST /v1?engine=<engine>_async on ecom.webscrapingapi.comsnapshot_idFinal JSON records

Web Unblocker async

WebScrapingAPI /v2 accepts these async fields:

ParameterBehavior
urlTarget page to scrape.
methodTarget request method.
bodyTarget request body for body-capable requests.
country2-letter country code for geotargeting.
render_js=1Enables JavaScript rendering.

Use country for async geotargeting. state, city, and asn are not supported for async /v2 submission.

SERP async

The google_async engine submits a SERP request and returns a response_id.

ParameterBehavior
SERP engine fieldsAdded to the target search request.
device=mobileRequests mobile SERP results.
device=tabletRequests tablet SERP results.
response=jsonRequests structured JSON where supported.
countryApplies country-level geotargeting when present.

Async batch engines

These engines submit a batch input array and return a snapshot_id:

EngineRequest modeInput
bestbuy_asyncAsync batchBest Buy product URLs or keyword inputs; see Best Buy Async.
booking_asyncAsync batchbody array; type selects hotel, hotel pricing, or search inputs.
chatgpt_asyncAsync batchbody array of prompt inputs.
nordstrom_asyncAsync batchbody array of Nordstrom product URLs.
target_asyncAsync batchTarget product or discovery inputs; see Target Async.
wayfair_asyncAsync batchbody array of Wayfair product URLs.

Poll the returned snapshot_id until the result is ready. Completed responses are returned as JSON records.

Batch submit example

curl --request POST \
  "https://ecom.webscrapingapi.com/v1?api_key=YOUR_API_KEY&engine=bestbuy_async&type=product" \
  --header "Content-Type: application/json" \
  --data '[
    {
      "url": "https://www.bestbuy.com/product/apple-airpods-pro-3-wireless-active-noise-cancelling-earbuds-with-heart-rate-sensing-feature-white/JJGCQLYK5F"
    }
  ]'

Response:

{
  "snapshot_id": "s_m4x7enmven8djfqak"
}

Poll:

curl --get "https://ecom.webscrapingapi.com/v1" \
  --data-urlencode "api_key=YOUR_API_KEY" \
  --data-urlencode "snapshot_id=s_m4x7enmven8djfqak"

Finished results remain available for a limited download window. Poll promptly and persist completed results in your own storage.

On this page