Checking whether a web page exists with the HEAD request

You would like to check the existence of a web page without downloading the HTML content. This means that we need to send a get HEAD request with a browser client. According to Wikipedia, the HEAD request asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.

How to do it...

We would like to send a HEAD request to http://www.python.org. This will not download the content of the home page, rather it checks whether the server returns one of the valid responses, for example, OK, FOUND, MOVED PERMANENTLY, and so on.

Listing 4.6 explains checking a web page with the HEAD request as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook -- Chapter - 4 
# This program requires Python 3.5.2 or any later version 
# It may run on any other version with...