Changing the user agent

Many websites use a user agent string to identify the browser and serve it accordingly. As we are using urllib to access the website, it won't recognize this user agent and may behave in strange ways or fail. So, in this case, we could specify the user agent for our requests.

How to do it...

We use custom user agent string in the request as following:

1. First, import the required modules:

>>> import urllib.request  

2. Then define the user agent we plan to specify for the request:

>>> user_agent = ' Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0'  

3. Set up the headers for the request:

>>> headers = {'User-Agent': user_agent...