Basic API Requests

WebScrapingAPI is not limited to GET requests. You can now easily scrape the internet by sending POST, PUT or PATCH requests as well.

Only basic understanding of how the web works is required to scrape the web using WebScrapingAPI. Our team made it so that anyone can extract HTML from almost any website, without having to write an entire application.

The url parameter

The url parameter has to be encoded in order for your request to pass!

Before diving into the more advanced features of WebScrapingAPI, it is important to understand how the API works. In order to scrape a web page using our services, simply append the URL of the page you want to scrape to the API’s base endpoint, along with your API key.

The url parameter represents the webpage that you are targeting and is required to make a request on our API. Failing to pass this parameter will result in 400: Bad Request.

This being said, it only takes two parameters to extract the full HTML code of the page you want to scrape using WebScrapingAPI: api_key and url.

Supported HTTP Methods

Whilst GET requests are quite common in web scraping, more advanced users may want to send POST or PUT requests to scrape forms or even other APIs. Currently, WebScrapingAPI offers support for the following HTTP requests:

• GET requests

• POST requests

• PUT requests

• PATCH requests