Handling change
Change is constant, as contradictory as that sounds. The architecture, data model, and technology will constantly evolve over time. You will need to decide how to handle change in order to keep flexibility in your data storage and processing architecture. You will want to use tools that are decoupled from each other to allow future analytics to be easily integrated into the data processing stream.
Fortunately, components in the Hadoop ecosystem were intentionally designed to be decoupled from each other. They allow the mixing and matching of components and were built to be extensible for new frameworks not even invented yet. Cloud infrastructures allow for easy testing and incorporation of new technologies and software.
You will need to design a process that takes your analytics and data processing code from experimentation, to development, to production. This holds true for your overall infrastructure as well. A common practice is to keep three separate environments: one for...
