Async Request

Web scraping asynchronously via GET requests requires users to pass three mandatory parameters to our Async API’s endpoint:

  • api_key - used to authenticate the request on our servers

  • url - used to indicate the targeted URL that you want to scrape

  • async - set to true to instruct the API to process the request asynchronously and immediately return a snapshot_id

The most basic URL example for an authorised request on our API is:

https://api.webscrapingapi.com/v2?api_key=<YOUR_API_KEY>&url=<TARGETED_URL>&async=true

This call will immediately return a snapshot_id, which you can then pass to our Snapshot API to fetch the results once they’re ready.

GET Request Integration Example

Basic Async Request Example

GET https://api.webscrapingapi.com/v2?api_key=<YOUR_API_KEY>&url=https://webscrapingapi.com/&async=true

Please make sure to update <YOUR_API_KEY> with your own API key. Find out how to obtain an API key.

The targeted website in this example is https://webscrapingapi.com/.

Query Parameters

Name
Type
Description

api_key

String

<YOUR_API_KEY>

url

String

https://webscrapingapi.com/

Below are the integration examples of the URL we have discussed earlier:

Response Example

Retrieving Results Using the Snapshot API

Once you've submitted a request to the Async API and received a snapshot_id, you can use the Snapshot API to retrieve the processed results. This allows you to fetch data asynchronously once it's ready, ensuring smooth and efficient operation.

We will use following URL as an example of getting results from a snapshot_id

When the data is ready, the API will return a JSON response containing the scraped results. Here's an example of the response structure:

Response Example

Last updated