Free and open source web crawling framework, written in Python
- Familiar with Scrapy installation
- Familiar with Scrapy functionality
- Able to scrape web pages for specific data
- Basic Python
- Some experience using Python
- Experience installing Python libraries
Scraping data from webpages can be a tedious job. But it doesn’t have to be.
With Scrapy, you can scrape using XPath or CSS. With the large number of examples from both techniques, you’re sure to find a solution that fits for you.
Whether your targeting data on a single page or multiple, Scrapy can handle the job. No matter if the data is within a list, you can scrape specific patterns right out of the list. Building up your specific Scrapy job isn’t a difficult task.
Scrapy is a Python library. If you’re familiar with Python, XPath or CSS, you’ll feel right at home using Scrapy.
At the end of this course, you will understand:
– what Scrapy is used for
– how to install it
– how to use Scapy
In summary, you’ll be able to target specific elements on a webpage, whether the element is stand along or in a list. Then you can retrieve a group of those elements or just one. This technique allows you to pull down specific types of data.
The course ends with a project to help solidify what you’ve learned. There is a full walk through included with the project solution.
Content retrieved from: https://www.udemy.com/python-scrapy/.