Search & SERP data

SERP data collection, built for how search works now

When bulk retrieval disappeared and JavaScript rendering became mandatory, the cost of collecting search results at depth jumped for everyone. Zyte's coordinated pagination and residential proxy infrastructure absorbs that complexity. Your unit economics don't collapse every time the rules change.

The basics

What is SERP data?

A search engine results page is the ranked list of organic results, ads, and rich features (AI Overviews, Knowledge Panels, local packs) that a search engine returns for a query. SERP data is a structured record of that page: which URLs rank where, what snippets appear, and what AI-generated content is shown. It is collected from live search queries, normalized into a consistent schema, and delivered as structured JSON. Because the same keyword returns different results depending on geo, device, language, and time, the value is not just in what ranks. It's in the context that surrounds it.
Who uses it

Use cases across industries

The same data type, put to work differently. Ordered by how directly it applies.

SEO & rank-tracking platforms

Daily keyword rank tracking and visibility monitoring across regions, devices, and SERP feature types.
Millions of keywords tracked daily

Competitive intelligence

Share-of-voice analysis, competitor ranking history, and SERP feature ownership across an entire category.
Full SERP snapshot, not just rank

Enterprise data teams

SERP data as an upstream signal feeding marketing attribution, demand forecasting, and brand monitoring.
Delivered as structured JSON feed

AI search visibility

Monitoring how brands appear in AI Overviews — which sources are cited, how descriptions are framed, and where competitors are favoured.
AI Overview content captured

Content & SEO agencies

Keyword gap analysis, content opportunity research, and before/after ranking validation for client campaigns.
Multi-client, multi-region coverage

Full-page extraction pipelines

SERP as a gateway: find ranked URLs, then follow them with Zyte API's extraction layer — one platform, one pipeline.
No vendor switching
The hard part

Why SERP data is hard to collect at scale

The problem is rarely a single keyword. It is collecting millions of them reliably, at depth, while the rules of collection keep changing underneath you.

Bulk retrieval parameters were removed

A single parameter once returned 100 results in one request. Its removal meant that the same depth now requires up to ten sequential paginated calls — each one a new browser session, a new proxy, a new opportunity to be blocked. For a platform tracking hundreds of thousands of keywords to page 10, that is not a theoretical cost increase. It is a real one, and it landed overnight.

JavaScript rendering is now mandatory

Modern SERPs, especially those serving AI Overviews, don't exist in the raw HTML. They render client-side, which means a plain HTTP request returns a shell, not a page. Reliable collection now requires a real browser with a residential proxy behind it. That is a fundamentally more expensive operation than what most pipelines were built to do.

Anti-bot enforcement scales with your volume

The more you collect, the more you look like an automated client. Search engine defences are dynamic: what works at 10,000 requests a day may fail at 10 million. Teams that built their own proxy rotation find that thresholds shift without notice, turning a working pipeline into a stream of failed requests and CAPTCHAs. The only durable answer is infrastructure that adapts faster than the blocks.

AI Overviews changed what a SERP is

LLM-generated answer blocks now appear above organic results for many queries. Teams that don't capture them are tracking an incomplete picture — and missing the placement their clients increasingly care about most. Zyte returns AI Overview content within the response HTML so your pipeline sees the full page, not just the links beneath it.
The cost of getting it wrong

What poor SERP data quietly costs the business

When bulk retrieval parameters were removed and rendering requirements tightened, teams that absorbed the change quietly are still paying for it. Bad SERP data doesn't announce itself. It shows up later, in a rank report that was already wrong, or a margin that's been silently compressing for months.
Margin pressure
10×
The cost per keyword for a 10-page deep pull compared to a single page, when each page is billed independently at a flat rate. Most SERP APIs do exactly this.
Silent data loss
1 in 6
A partial SERP failure — one page of ten coming back empty — looks like a complete dataset until you check rank position distributions. Dashboards stay green while the data degrades.
Engineering overhead
2–4 wks
The typical sprint cost every time pagination logic, proxy configuration, or retry behaviour needs to be rebuilt after an enforcement change. It is not a one-time cost.
Strategic blindspot
~30%
Of tracked queries now return an AI Overview. Teams without structured capture of these blocks are monitoring what used to matter, not what their customers actually see.
See the Schema

The request you send and the data that comes back

Pick the standard schema or a custom one mapped to your model, and read the response as a table or JSON.

REQUEST
POST https://api.zyte.com/v1/extract

{

  "url": "https://example-search.com/search?q=best+enterprise+SEO+platform",

  "serp": true

}
RESPONSE
Standard Search & SERP schema
Field
Type
Example
url
string
https://example-search.com/search?q=best+enterprise+SEO+platform[https://example-search.com/search?q=best+enterprise+SEO+platform]
keyword
string
best enterprise SEO platform
geo
object
{ … }
country
string
US
language
string
en
collectedAt
string
2026-07-02T08:00:00Z
pageDepth
integer
10
resultsTotal
integer
100
serpFeatures
array
["aiOverview", "organicResults"]
aiOverview
object
{ ... }
captured
string
captured
organicResults
array
[{...}]
rankPosition
number
1
url
string
seoplatform.com/enterprise [http://seoplatform.com/enterprise]
title
string
Best Enterprise SEO Platform 2026
snippet
string
Compare the top enterprise SEO platforms by features, pricing, and support.
Reviews

What our users say

I have been working with Zyte's team for the last few months, and their team is fantastic. I appreciate their development speed and quality, and they run a very robust platform, producing very satisfactory results. I love the ease of the initial setup with Zyte, as they took care of all the development, and we only needed to communicate what data we needed and set up the necessary processes on our end.

David P.

Frequently asked questions

Does Zyte's /search endpoint replace what bulk retrieval parameters used to do?

Yes, and more precisely. Zyte's coordinated pagination returns up to 100 organic results from a single API call — one request, ten pages fetched and assembled internally. Pages 2–10 share the session and proxy infrastructure established on page 1, so they cost less to run and less to buy. You get the same depth you had before at a materially better unit cost than ten independent requests would produce.

Do you handle JavaScript-rendered SERPs, including AI Overviews?

Yes. Zyte uses browser rendering by default for SERP collection. AI Overview content is returned inside the response HTML for every query where it is served — your pipeline sees the full page, not the pre-render shell.

What formats and delivery methods do you support?

Zyte delivers data in JSONL, Parquet, CSV, and custom formats, via S3, SFTP, API, or direct warehouse integration. Format and delivery cadence are scoped per project.

How fresh can SERP data be?

Rank data is typically delivered daily or on-demand; freshness is scoped per use case based on how often the relevant SERPs actually change. Real-time and event-triggered collection is available where needed.

How do you handle layout changes that break extraction?

Zyte validates every collection run against your agreed schema. Layout changes trigger automatic detection and escalation before gaps reach your data.

What about anti-bot measures and blocks?

When a request is blocked, Zyte automatically re-routes and retries with a different proxy, session, or rendering approach. Persistent blocks escalate to the team running your pipeline rather than silently failing.

What formats and delivery methods do you support?

Structured JSON via the API, with delivery to your preferred destination. Custom schema mapping is available for teams with existing data models.

How long does setup take?

Self-serve access starts immediately via Zyte API. Enterprise agreements with custom schema, volume pricing, and dedicated support are typically scoped and live within a few weeks.

Can I see a sample before committing?

Yes. Talk to a data specialist to request a sample run on the keywords and markets you care about.