Collecting signatures of web servers

Nmap is a de facto tool for information gathering, and the variety of tasks that can be done with the Nmap Scripting Engine is simply remarkable. The popular service "ShodanHQ" (http://shodanhq.com) offers a database of HTTP banners, which is useful for analyzing the impact of vulnerabilities. Its users can find out the number of devices that are online, by country, which are identified by their service banners. ShodanHQ uses its own built-in house tools to gather its data, but Nmap is also perfect for this task.

In the following recipe, we will see how to scan indefinitely for web servers, and collect their HTTP headers by using Nmap.

How to do it...

Open your terminal and enter the following command:

$ nmap -p80 -Pn -n -T4 --open --script http-headers,http-title --script-args http.useragent="A friend web crawler (http://someurl.com)",http-headers.useget -oX random-webservers.xml -iR 0

This command will launch an instance of Nmap that will run indefinitely...