Forcing Timeouts

BrowserAPI allows you to adjust the timeout value in order to obtain lazy loaded content.

When dealing with websites that require longer load time for the full DOM, BrowserAPI's default timeout value may result in returning only the portion of the DOM that was gathered until the timeout was triggered.

In that particular case, if you notice that the response from our API only includes fractions of the DOM, you may want to tweak the load time by passing the timeout=$value parameter.

Your full GET request should then be sent to the following address:

https://api.webscrapingapi.com/v1?api_key=<YOUR_API_KEY>&url=<TARGETED_URL>&timeout=$value

Timeouts Integration Examples

curl --request GET --url "https://api.webscrapingapi.com/v1?api_key=<YOUR_API_KEY>&url=https://httpbin.org/get&timeout=200"

Important! The url parameter has to be encoded. ( i.e. &url=https%3A%2F%2Fwww.webscrapingapi.com%2F )

Response Example
{
    "status": "Failure",
    "status_code": 422,
    "created_at": "2022-04-26T11:57:23.242Z",
    "processed_at": "2022-04-26T11:57:23.739Z",
    "time_taken": {
        "total": 0.701,
        "scraping": 0.202,
        "setup_worker": 0.403
    },
    "error": "The target page took more than 0.2 seconds to load, the website might be down. Retry the request or increase the value of 'timeout' parameter.",
    "page_content": null
}

Last updated