Using search methods to scrape information from a web page
In Chapter 2, Creating a BeautifulSoup Object, we discussed the fetching of information related to all books published by Packt Publishing available at the link http://www.packtpub.com/books. In the following exercise, we will use Beautiful Soup searching methods to scrape all books from the preceding URL. We will scrape the name of the book, published date, and price from the preceding URL.
The first step involved in this exercise is to analyze the HTML document and to understand its logical structure. This will help us to understand how the required information is stored within the HTML document.
We can use the Google Chrome developer tools to understand the logical structure of the page. Let us do that by performing the following steps:
- Open the page in the Google Chrome browser.
- Right-click on it and select the Inspect Element option.
- Now, we can see the Developer tools window at the bottom of the page.
The previous URL content under...
