Starting from this chapter, we will be exploring scraping-related tools and techniques, as we will also be deploying some scraping code. Features related to web exploration, Python libraries, element identification, and traversing are the major concepts we have learned about so far.
Web scraping is often a challenging and long process that requires an understanding of how the website is performing. A basic ability to understand and identify the backends or tools that are used to build a website will assist in any scraping task. This is also related to a process known as reverse engineering. For more information on such tools, please refer to Chapter 3, Using LXML, XPath, and CSS Selectors, and the using web browser developer tools for accessing web content section. In addition to this, identifying the tools for traversing and manipulating...
