Collecting signatures of web servers
Nmap is an amazing tool for information gathering, and the variety of tasks that can be done with NSE is simply remarkable. The popular service ShodanHQ (https://www.shodan.io/) offers a database with a nice GUI of HTTP banners, which is useful for analyzing the impact of vulnerabilities. Its users can find out the number of devices that are online by country, which are identified by their service banners and IP geolocation. ShodanHQ uses its own built-in house tools to gather its data, but Nmap can easily be used for this task and take advantage of NSE.
In this recipe, we will see how to scan indefinitely for web servers and collect their HTTP headers with Nmap.
How to do it...
Open your terminal and enter the following command:
$ nmap -p80,443 -Pn -T4 --open --script http-headers,http-title,ssl-cert --script-args http.useragent="A friendly web crawler (http://calderonpale.com)",http-headers.useget -oX random-webservers.xml...