API Parameters

Discover the full power of WebScrapingAPI by passing advanced parameters to your requests.

Parameters are being used to customise each request to the API. Understanding how to use these parameters will allow you to reach the full potential of the WebScrapingAPI and better interact with the targeted URL.

Below is a list of all the parameters available in our API:

#1: Required Parameters

ParameterTypeDescriptionMore

api_key Required

string

authenticate

url Required

string

Specify the URL of the web page you would like to scrape.

IMPORTANT: Please make sure you URL encode the value of this parameter!

#2: Optional Parameters

ParameterTypeDescriptionMoreDefault Value

render_js

int

Set to 0 (off) or 1 (on, default) depending on whether or not to render JavaScript on the target web page. JavaScript rendering is done by using a browser.

The default value for render_js will be decided based on your plan. If your plan supports javascript rendering then the default value will be true. Otherwise the default value will be false.

proxy_type

string

Set to datacenter or residential (default) depending on whether proxy type you want to use for your scraping request.

Please note that a single residential proxy API request is counted as 10 API calls when render_js is off and 25 API calls when the render_js is on.

The default value for proxy_type will be decided based on your plan. If your plan supports residential proxies then the default value will be residential. Otherwise the default value will be datacenter.

stealth_mode

int

Set to 0 (off, default) or 1 (on) to enable custom evasion techniques, allowing you to scrape websites with anti-bot systems. Using this parameeter is counted as 75 API calls.

country

string

Specify the 2-letter code of the country you would like to use as a proxy geolocation for your scraping API request. Supported countries differ by proxy type, please refer to the Proxy Locations section for details.

timeout

int

Specify the maximum timeout in milliseconds you would like to use for your scraping API request. In order to force a timeout, you can specify a number such as 1000.

This will abort the request after 1000ms and return whatever HTML response was obtained until this point in time. The maximum value for this parameter is 60000.

device

string

Set desktop (default), mobile or tablet, depending on whether the device type you want to your for your scraping request.

wait_until

string

Specify the option you would like to use as conditional for your scraping API request.

*Can only be used when the parameter render_js=1 is activated.

wait_for

int

Some websites may use javascript frameworks that may require a few extra seconds to load their content.

This parameters specifies the time in milliseconds to wait for the website. Recommended values are in the interval 5000 - 10000.

*Can only be used when the parameter render_js=1 is activated.

wait_for_css

string

Specify a CSS selector and the API will wait 10 seconds (the default value of the timeout parameter) until the selector appears.

*Can only be used when the parameter render_js=1 is activated.

window_width

int

Adjust the width of the web browser's Viewport. *Can only be used when the parameter render_js=1 is activated.

window_height

int

Adjust the height of the web browser's Viewport. *Can only be used when the parameter render_js=1 is activated.

screenshot

int

Set to 1 to get the scraped website as a screenshot.

*Can only be used when the parameter render_js=1 is activated.

screenshot_options

object

Of type object , this parameter is used to set the screenshot to return custom window sizing, full page or single CSS selector.

*Can only be used when the parameter screenshot=1 is activated

extract_rules

object

Get the scraped website based on various extraction rules.

json_response

int

Have the API return an JSON object instead of the plain HTML scraped from the URL.

json_dom

int

Get the scraped website in JSON format. It can be set to 0 (default) or 1.

js_instructions

object

Perform JavaScript instructions before obtaining the scraped website.

disable_stealth

int

Disable the stealth plugin which is enabled by webscrapingapi by default. It can be set to 1 (to disable the stealth plugin) or 0 (to keep the stealth plugin enabled).

*Can only be used when the parameter render_js=1 is activated.

block_resources

string

Block web browser's resources (i.e. block_resources=media,js)

block_ads

int

Block ads in headless mode. By default WebScrapingAPI renders and returns ads inside the response. It can be set to 1 (to enable ads blocking) or 0 (to keep ads inside the DOM). *Can only be used when the parameter render_js=1 is activated.

block_trackers

int

Block trackers in headless mode. By default WebScrapingAPI does not block trackers. It can be set to 1 (to enable trackers blocking) or 0 (to allow trackers). *Can only be used when the parameter block_ads=1 is activated.

session

int

Set depending on whether or not to use the same proxy address to your request.

keep_headers

int

Specify whether or not to keep the original request headers in order to pass through custom headers. In order to only use the headers that are specified, set the parameter keep_headers=0.

return_page_source

int

Reetrieve the unaltered HTML from the server, before JavaScript rendering. *Can only be used when the parameter render_js=1 is activated.

cookies

object

Add custom cookies to your request. *It only works when the parameter render_js=1 is activated. If you want to add custom cookies when render_js=0 follow this link

Last updated