Batch processing has been used for many years and there has been some common design problems encountered during batch processing implementation. People have some common design patterns to solve such problems. In this section, we will basically discuss some of the common design patterns and how we solve design problems using common techniques.
Common batch processing pattern
Slowly changing dimension
Slowly Changing Dimension (SCD) refers to the concept where some or most part of the data changes at irregular intervals. There are multiple SCD types available and each have different implementations in Hadoop. We will talk about each type and how to deal with it. We will look into Type 1 and Type 2, which are commonly used slowly...
