Network intercept
Discover and capture website network patterns
Navigate any website’s network traffic to analyze its communications to outside resources. Optimize your scraping by going for the source of information.
Explore how websites collect and source their data
All requests made using the Zyte API with Browser rendering provide access to the network intercept feature in the API debugger. This feature enables you to intercept headers, cookies, and all the requests a website makes to gather content from external sources, such as APIs, other websites, and JSON files.
Intercept patterns after browser actions
The Zyte API includes a powerful headless browser allowing network capture before and after mimicking user actions such as click, scroll, and type.
Replicate requests in your spiders
By understanding how a website sources data, you can directly replicate requests to APIs and external resources in your spiders. This approach saves time and reduces costs associated with browser rendering and screenshots.
Advanced filtering options on intercepted patterns
Utilize advanced filtering options to process only the pattern types that matter, streamlining your network analysis and saving time.
Getting Started with Zyte API
Use this handy tutorial to get started with Zyte API in minutes.
from base64 import b64decode
import requests
api_response = requests.post(
'https://api.zyte.com/v1/extract',
auth=('YOUR_API_KEY', ''),
json={
'url': 'https://toscrape.com',
'httpResponseBody': True,
},
)
http_response_body: bytes = b64decode(
api_response.json()['httpResponseBody']
)# Prerequisite: install scrapy-zyte-api and configure it in transparent mode
# https://github.com/scrapy-plugins/scrapy-zyte-api
from scrapy import Spider
class ToScrapeSpider(Spider):
name = "toscrape_com"
start_urls = ["https://toscrape.com"]
def parse(self, response):
http_response_text: str = response.textconst axios = require('axios')
axios.post(
'https://api.zyte.com/v1/extract',
{
url: 'https://toscrape.com',
httpResponseBody: true
},
{
auth: { username: 'YOUR_API_KEY' }
}
).then((response) => {
const httpResponseBody = Buffer.from(
response.data.httpResponseBody,
'base64'
)
})$client = new GuzzleHttp\Client();
$response = $client->request('POST', 'https://api.zyte.com/v1/extract', [
'auth' => ['YOUR_API_KEY', ''],
'headers' => ['Accept-Encoding' => 'gzip'],
'json' => [
'url' => 'https://toscrape.com',
'httpResponseBody' => true,
],
]);
$data = json_decode($response->getBody());
$http_response_body = base64_decode($data->httpResponseBody);curl \
--user YOUR_API_KEY: \
--header 'Content-Type: application/json' \
--data '{"url": "https://toscrape.com", "httpResponseBody": true}' \
--compressed \
https://api.zyte.com/v1/extractOnly pay for what you use
We know your projects are unique, we don’t force you into a “package”. Instead, we price Zyte API around you. Below we’ve shown some examples, based on extraction complexity.
Unrendered HTTP requests - price per 1000 successful requests
| Website Tier | Website Distribution | Technology | PAYG | $100* | $200* | $350* | $500* |
|---|---|---|---|---|---|---|---|
| 1 | $0.13 | $0.10 | $0.08 | $0.07 | $0.06 | ||
| 2 | $0.23 | $0.17 | $0.14 | $0.12 | $0.11 | ||
| 3 | $0.43 | $0.32 | $0.26 | $0.22 | $0.21 | ||
| 4 | $0.70 | $0.52 | $0.42 | $0.36 | $0.33 | ||
| 5 | $1.27 | $0.95 | $0.76 | $0.65 | $0.60 |
Data centre proxies
Residential proxies
Compute
PAYG (Pay As You Go - up to $100/month)
*monthly commitment
Rendered browser requests - price per 1000 successful requests
| Website Tier | Website Distribution | Technology | PAYG | $100* | $200* | $350* | $500* |
|---|---|---|---|---|---|---|---|
| 1 | $1.00 | $0.75 | $0.60 | $0.52 | $0.47 | ||
| 2 | $2.00 | $1.50 | $1.20 | $1.03 | $0.95 | ||
| 3 | $4.00 | $3.00 | $2.40 | $2.06 | $1.89 | ||
| 4 | $7.99 | $5.99 | $4.79 | $4.12 | $3.79 | ||
| 5 | $15.98 | $11.98 | $9.58 | $8.25 | $7.58 |
Data centre proxies
Residential proxies
Compute
PAYG (Pay As You Go - up to $100/month)
*monthly commitment