Links

API Parameters

Discover the full power of WebScrapingAPI by passing advanced parameters to your requests.
Parameters are being used to customise each request to the API. Understanding how to use these parameters will allow you to reach the full potential of the WebScrapingAPI and better interact with the targeted URL.
Below is a list of all the parameters available in our API:

#1: Required Parameters

Parameter
Type
Description
More
api_key Required
string
authenticate
learn
url Required
string
Specify the URL of the web page you would like to scrape.
IMPORTANT: Please make sure you URL encode the value of this parameter!
learn

#2: Optional Parameters

Parameter
Type
Description
More
Default Value
render_js
int
Set to 0 (off) or 1 (on, default) depending on whether or not to render JavaScript on the target web page. JavaScript rendering is done by using a browser.
learn
The default value for render_js will be decided based on your plan. If your plan supports javascript rendering then the default value will be true. Otherwise the default value will be false.
proxy_type
string
Set to datacenter or residential (default) depending on whether proxy type you want to use for your scraping request.
Please note that a single residential proxy API request is counted as 10 API calls when render_js is off and 25 API calls when the render_js is on.
learn
The default value for proxy_type will be decided based on your plan. If your plan supports residential proxies then the default value will be residential. Otherwise the default value will be datacenter.
stealth_mode
int
Set to 0 (off, default) or 1 (on) to enable custom evasion techniques, allowing you to scrape websites with anti-bot systems. Using this parameeter is counted as 75 API calls.
learn
country
string
Specify the 2-letter code of the country you would like to use as a proxy geolocation for your scraping API request. Supported countries differ by proxy type, please refer to the Proxy Locations section for details.
learn
timeout
int
Specify the maximum timeout in milliseconds you would like to use for your scraping API request. In order to force a timeout, you can specify a number such as 1000.
This will abort the request after 1000ms and return whatever HTML response was obtained until this point in time. The maximum value for this parameter is 60000.
learn
device
string
Set desktop (default), mobile or tablet, depending on whether the device type you want to your for your scraping request.
learn
wait_until
string
Specify the option you would like to use as conditional for your scraping API request.
*Can only be used when the parameter render_js=1 is activated.
learn
wait_for
int
Some websites may use javascript frameworks that may require a few extra seconds to load their content.
This parameters specifies the time in milliseconds to wait for the website. Recommended values are in the interval 5000 - 10000.
*Can only be used when the parameter render_js=1 is activated.
learn
wait_for_css
string
Specify a CSS selector and the API will wait 10 seconds (the default value of the timeout parameter) until the selector appears.
*Can only be used when the parameter render_js=1 is activated.
learn
window_width
int
Adjust the width of the web browser's Viewport. *Can only be used when the parameter render_js=1 is activated.
learn
window_height
int
Adjust the height of the web browser's Viewport. *Can only be used when the parameter render_js=1 is activated.
learn
screenshot
int
Set to 1 to get the scraped website as a screenshot.
*Can only be used when the parameter render_js=1 is activated.
learn
screenshot_options
object
Of type object , this parameter is used to set the screenshot to return custom window sizing, full page or single CSS selector. Available options:
{ width:$value, height:$value } { full_page: 1 } { screenshot_selector:$selector }
{ return_html: 1 } *Can only be used when the parameter screenshot=1 is activated.
learn
extract_rules
object
Get the scraped website based on various extraction rules.
learn
json_response
int
Have the API return an JSON object instead of the plain HTML scraped from the URL.
learn
json_dom
int
Get the scraped website in JSON format. It can be set to 0 (default) or 1.
learn
js_instructions
object
Perform JavaScript instructions before obtaining the scraped website.
learn
disable_stealth
int
Disable the stealth plugin which is enabled by webscrapingapi by default. It can be set to 1 (to disable the stealth plugin) or 0 (to keep the stealth plugin enabled).
*Can only be used when the parameter render_js=1 is activated.
learn
block_resources
string
Block web browser's resources (i.e. block_resources=media,js)
learn
block_ads
int
Block ads in headless mode. By default WebScrapingAPI renders and returns ads inside the response. It can be set to 1 (to enable ads blocking) or 0 (to keep ads inside the DOM). *Can only be used when the parameter render_js=1 is activated.
learn
block_trackers
int
Block trackers in headless mode. By default WebScrapingAPI does not block trackers. It can be set to 1 (to enable trackers blocking) or 0 (to allow trackers). *Can only be used when the parameter block_ads=1 is activated.
learn
session
int
Set depending on whether or not to use the same proxy address to your request.
learn
keep_headers
int
Specify whether or not to keep the original request headers in order to pass through custom headers. In order to only use the headers that are specified, set the parameter keep_headers=0.
learn
return_page_source
int
Reetrieve the unaltered HTML from the server, before JavaScript rendering. *Can only be used when the parameter render_js=1 is activated.
learn