Architecture Principles
Thus far we have written about the importance of establishing your organization’s mission, vision, best practices, and patterns, and general corporate principles; however, there are specific principles related to the architecture when developing a future-proof data engineering solution that will have to be addressed as we move forward. These principles carry with them the underlying corporate and IT values that will help keep a solution effective in the future. Tools, vendors, and various cloud offerings will morph to embrace new machine intelligence capabilities.
In this chapter, you will be provided with the foundation for developing principles. You will also be provided with a number of principles that are relevant to the task of building a future-proof data factory. Lastly, you will be provided with an abundant number of references for additional reading needed to flesh out a common understanding of challenges that will be presented as you form an architecture that will withstand the test of time.
We begin with an overview of the technology landscape and the way businesses jumpstart their efforts with research firm findings. These non-academic sources are highly compensated for the information they provide, and that information often defines how the herd is moving. We use the term herd because those who wish to lead the field of their peers are those who have the qualifications, knowledge, and experience to advance cutting edge technologies without excess cost or outright failure.
