Introducing big data

There has been a lot of excitement about big data and the kind of skills involved with it. Before beginning this chapter, it is important to define what big data is and how you can work with big data to gather useful information. How different is big data from the kind of data we come across everyday, say news stories, reports, literature, or even audio?

Big data is actually data captured at a high velocity, and it accumulates in such large quantities that it takes up terabytes or petabytes of storage. Common software tools are inadequate for capturing, processing, maintaining, and managing such data with a short period of tolerance. Analytical tools are applied on these large datasets to uncover the information and relationships that could possibly be used for forecasting or other analytical activities.

With the advent of cloud computing technologies, big data has become increasingly commonplace. Massive amounts of information can be stored in the cloud at lower costs...