Explore resources by topic or category

Discover how Zyte’s open-source libraries like ClearHTML, Extruct, Chomp.js, and more simplify web data extraction and processing.

Discover the strengths and limitations of Selenium, Puppeteer, and Playwright for web scraping at scale.

Here are four essential Scrapy plugins we use to build efficient web crawlers for our customers.

Web scraping tools save hours of work by automating data extraction, testing web applications, and performing repetitive tasks.

In the first part, we discussed a template to define the clear purpose of your web scraping system that can help you design your crawlers better and prepare you for the uncertainty involved in a large scale web scraping project.

It was 6 years ago when Zyte released Dateparser, an open source library that parses human-readable dates, and in October 2020 we released version 1.0.0, a very important milestone.

When crawling the web, there’s always a speed limit. A spider can't fetch faster than the host willing to send the pages.

As a python developer at Zyte (formerly Scrapinghub), I spend a lot of time in the Scrapy shell.

If you’ve been using Scrapy for any period of time, you know the capabilities a well-designed Scrapy spider can give you.

If you know anything about Zyte , you know that we are obsessed with data quality and data reliability.

Absolutely not! Website changes (sometimes very subtly), anti-bot countermeasures, and temporary problems often reduce the quality and reliability of our data.

This is a guest post from the folks over at Intoli, one of the awesome companies providing Scrapy commercial support and longtime Scrapy fans.