Introduction to Scrapy components and architecture

In this section, we will learn about Scrapy components and architecture. We will review Scrapy architecture and XPath expressions from Scrapy shell.

What is Scrapy?

Scrapy (https://scrapy.org/) is an open source collaborative platform that allows us to extract data from web pages used for a series of applications such as data mining, information processing, and historical registration.

This framework also allows us to expand its functionality and is portable because it is written in Python, which can be interpreted on Linux, Macintosh, and Windows systems.

Although the main objective of Scrapy is the extraction of data from web pages, this can also be used to extract data...