Explore resources by topic or category

The first rule of web crawling is you do not harm the website. The second rule of web crawling is you do NOT harm the website. We’re supporters of the democratization of web data, but not at the expense of the website’s owners.

Welcome to This Month in Open Source at Zyte! In this regular column, we share all the latest updates on our open source projects including Scrapy, Splash, Portia, and Frontera.

We eat our own spider food since Scrapy is our go-to workhorse on a daily basis. However, there are certain situations where Scrapy can be overkill and that’s when we use Parsel.

Many governments worldwide have laws enforcing them to publish their expenses, contracts, decisions, and so forth, on the web.

Welcome to the April Edition of Scrapy Tips from the Pros. Each month we’ll release a few tricks and hacks that we’ve developed to help make your Scrapy workflow go more smoothly.

Welcome to This Month in Open Source at Zyte! In this monthly column, we share all the latest updates on our open source projects including Scrapy, Splash, Portia, and Frontera.

Welcome to the February Edition of Scrapy Tips from the Pros. Each month we’ll release a few tips and hacks that we’ve developed to help make your Scrapy workflow go more smoothly.

Imagine your business depended heavily on a third party tool and one day that company decided to shut down its service with only 2 weeks notice. That, unfortunately, is what happened to users of Kimono Labs yesterday.