Python Web Scraping: Successfully scrape data from any website with the power of Python

Richard Penman

$14.99 ~~$21.99~~

3.8 (10 Ratings)

eBook Oct 2015 174 pages 1st Edition

Richard Penman

$14.99 ~~$21.99~~

3.8 (10 Ratings)

eBook Oct 2015 174 pages 1st Edition

What do you get with eBook?

Instant access to your Digital eBook purchase

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Preview Book

Description

The Internet contains the most useful set of data ever assembled, largely publicly accessible for free. However, this data is not easily reusable. It is embedded within the structure and style of websites and needs to be carefully extracted to be useful. Web scraping is becoming increasingly useful as a means to easily gather and make sense of the plethora of information available online. Using a simple language like Python, you can crawl the information out of complex websites using simple programming. This book is the ultimate guide to using Python to scrape data from websites. In the early chapters it covers how to extract data from static web pages and how to use caching to manage the load on servers. After the basics we'll get our hands dirty with building a more sophisticated crawler with threads and more advanced topics. Learn step-by-step how to use Ajax URLs, employ the Firebug extension for monitoring, and indirectly scrape data. Discover more scraping nitty-gritties such as using the browser renderer, managing cookies, how to submit forms to extract data from complex websites protected by CAPTCHA, and so on. The book wraps up with how to create high-level scrapers with Scrapy libraries and implement what has been learned to real websites.

Who is this book for?

This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved.

What you will learn

Extract data from web pages with simple Python programming
Build a threaded crawler to process web pages in parallel
Follow links to crawl a website
Download cache to reduce bandwidth
Use multiple threads and processes to scrape faster
Learn how to parse JavaScriptdependent websites
Interact with forms and sessions
Solve CAPTCHAs on protected web pages
Discover how to track the state of a crawl

What do you get with eBook?

Instant access to your Digital eBook purchase

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$26.99

Python Data Visualization Cookbook (Second Edition)

$48.99

Total $ 75.98

Filter reviews by

All

Amazon verified reviews

Tim Crothers Dec 04, 2015

Hands down the best resource I've found for practical examples of how to write web scrapers in Python. The author's style is very easy to read and very practical focused. He also clearly knows the subject inside and out and does a great job of not only showing you actual working code to do everything but also covers multiple approaches for different situations as well as key pitfalls to avoid.

Amazon Verified review

Bob Mar 23, 2017

I am a python guy and wanted to get into web scraping. This book was perfect for me. Very well written and great examples. Hope to see more from this author in the future.

Jean Marc QUERE Dec 04, 2015

If you need to collect (on a regular basis) data on a (or more) website(s), this book is for you. It explains how to load, extract, transform and store data in a convenient (and respectful) way. You will learn and practice this with Python. Prior to this, I had to wrote compiled programs and any change was boring. Now with Python, I can easily change my scripts without having to rebuild each program. The chapters about CAPTCHA and Scrapy were very useful for me... but don't tell it : I like to think I can do things that others can't :-)

Austin Miller Mar 26, 2018

Great resource!

Julian Cook Dec 29, 2015

This is a good up-to-date book on grabbing data from web sources..especially for python users who are not professional developers. I have found that any time you collect data, you normally end up either needing to auto-refresh the data, or you end up needing meta-data about the raw data. He has a good example, which is country statistics - something that you may need to 'normalize' other statistical data you are already working with.Typically python libs dealing with url fetching have very simple examples of the parser.parse(<html>"hello world"</html>) type, so you have no idea what to do with a real web page, like a 'needs login' page.The author, to his credit, does not tell you to download a 'magic' python library.In these cases he gives a thorough walk-through on how to research the structure or scripting of the page and then go about fetching it via python. Most of the book is in fact devoted to analysis, rather than action.The only parts that didn't seem particularly useful to me, were the chapters on creating crawlers and spiders. I don't see myself doing any of that and I don't see why an amateur or professional would use them. A professional would probably use Elasticsearch for instance.Other than that the book will probably be useful for a long time.

Python Web Scraping: Successfully scrape data from any website with the power of Python

What do you get with eBook?