The BigTable

If you decided to download every web page hosted on the planet, store a copy of it and later process every page to extract data from it, you'll find that your own laptop or desktop is not up to the task. It has barely enough storage to store every page. Usually, laptops these days come with 1 TB hard disk drives, and this seems to be quite enough for someone not much into video content such as movies.

Assuming that there are 2 billion websites, each with an average of 50 pages and each page around 250 KB in size, it sums up to around 23,000+ TB (or roughly 22 petabytes) which would need 23,000 such laptops to store all the web pages with a 1 TB hard drive in each.

Assuming the same statistics, if you are able to download at a whopping speed of 100 MBps, it would take you about 7 years to download the whole content to one such gigantic hard drive if you had one in your laptop.

Let's suppose that you somehow downloaded all the content in whatever time it took to do so...