Hadoop MapReduce

After installing Hadoop you can run its version of MapReduce quite easily. As we have seen, this amounts to writing your own versions of the map() and reduce() methods to solve the particular problem. This is done by extending the Mapper and Reducer classes defined in the package org.apache.hadoop.mapreduce.

For example, to implement the WordCount program, you could set your program up like the one shown in Listing 11-5.

Listing 11-5. WordCount program in Hadoop

The main class has two nested classes named WordCountMapper and WordCountReducer. These extend the corresponding Hadoop Mapper and Reducer classes, with a few details omitted. The point is that the map() and reduce() methods, that are to be written, are defined in these corresponding classes. This structure is what makes the Hadoop MapReduce framework an actual software framework.

Note that the Text class used in the parameter lists at lines 11 and 17 are defined in the org.apache.hadoop.io package.

This complete example...