The Scrapy tutorial part V: How to scrape details pages from lists
Developed by Pablo Hoffman and Shane Evans, Scrapy is an open-source python framework built specifically for web data extraction. With Scrapy spiders, you are able to download HTML, parse and process the data and save it in either CSV, JSON, or XML file formats.
This video shows how to scrape websites that are structured similarly to eCommerce, where there are lists of products and we have to visit each product page to get the data we need.
After watching this video, you will know:
- How to deal with multiple pages with different formats.
- Multiple callbacks per spider.
If you haven't yet, we recommend you to first watch part I, part II, part III, and part IV of our tutorial series.