Summary

In an era where organizations are aiming to do more with less, automation is quickly gaining a lot of attention. As CI/CD continues to grow and gain strength, it is set to become one of the most critical skills for modern data engineers. In most cases, the high cost of data engineers can only be justified if their skill set includes automation.

In many respects, adopting automation practices such as CI/CD is proving to be a lifesaver. Not only does automation take a lot of work off the data engineers' shoulders, but it also lowers costs by predictably performing repetitive iterations. On top of that, the built-in approval and fail fast mechanisms in CI/CD ensure team accountability and collaboration. If used wisely, adopting automation can ensure the predictable and seamless delivery of code and infrastructure components.

This is the last chapter of this book. I must admit that in the last 12 chapters, we have covered a lot of ground. We undertook the journey of...