Extraction Rules

Return selected text, HTML, attributes, tables, or nested data from a WebScrapingAPI request.

WebScrapingAPI can return selected parts of a page instead of the full HTML response. Pass a JSON-stringified extract_rules object to the /v2 endpoint.

curl --get "https://api.webscrapingapi.com/v2" \
  --data-urlencode "api_key=YOUR_API_KEY" \
  --data-urlencode "url=https://webscrapingapi.com" \
  --data-urlencode 'extract_rules={"title":{"selector":"h1","output":"text"},"links":{"selector":"a","output":"@href","all":1}}'

Query Parameters

Name	Required	Description
`api_key`	✅	Your WebScrapingAPI key.
`url`	✅	The target page URL.
`extract_rules`	✅	A JSON-stringified object that maps output keys to selectors or rule objects.

Rule Options

Option	Type	Description
`selector`	`string`	CSS selector or XPath expression. Required when the rule is an object.
`selector_type`	`string`	`css` or `xpath`. If omitted, WebScrapingAPI treats selectors that start with `/` as XPath and other selectors as CSS.
`output`	`string`	`text` by default. Use `html`, `@[attr]`, `table_json`, `table_array`, or a nested rule object.
`all`	`int`	Set to `1` to return all matching elements. Set to `0` to return only the first match.
`clean`	`int`	Set to `1` to trim leading and trailing whitespace and line breaks from text output.

You can use a shorthand string when text output from a selector is enough:

{
  "title": "h1"
}

Use rule objects when you need attributes, HTML, table parsing, or nested extraction:

{
  "title": {
    "selector": "h1",
    "output": "text"
  },
  "links": {
    "selector": "a",
    "output": "@href",
    "all": 1
  }
}

Response

The response contains one property for each rule key:

{
  "title": ["Transform Websites Into Data"],
  "links": ["/pricing", "/docs"]
}

If extract_rules is not a stringified object, WebScrapingAPI returns a validation error:

{
  "status": "Failure",
  "status_code": 400,
  "error": "Key `extract_rules` must be a stringified object."
}

Nested Data

Nested rules are useful for repeated cards, rows, or search result blocks. Select the outer item first, then define the fields to extract inside each match.

{
  "results": {
    "selector": ".result",
    "all": 1,
    "output": {
      "title": {
        "selector": "h2",
        "output": "text"
      },
      "url": {
        "selector": "a",
        "output": "@href"
      }
    }
  }
}

URL-encode the full extract_rules value when you build the request manually. Using curl --data-urlencode, URLSearchParams, or a request library avoids broken JSON in the query string.

Extraction Rules

Query Parameters

Rule Options

Response

Nested Data

On this page