Is big data for me?

Perhaps you might be working on a file that is hundreds of megabytes in size, and your current data analysis tool is performing too slowly. Since big data typically involves terabytes or gigabytes worth of storage, you might be wondering, Is Apache Hadoop for me?

Note that Hadoop implements one general computation by mapping every single entity on your data, and then performing some reduction computation to add up the individual parts. You might be able to achieve the same grouping and counting function using programming languages, such as SQL and Python. In addition, writing the code allows you to express the computational flow more easily.

Or you might want to consider migrating your data analysis tools to pandas or R. They are very powerful and able to handle gigabytes of data when coded efficiently with no memory leaks. Most commercial SQL servers are up to the task. Besides, memory and storage costs are so affordable that you could be performing heavy computations...