Test-Driven Development, Code Health, and Maintainability

In this chapter, we are going to look at some software development best practices and learn how to apply them to data engineering. The topics covered in the chapter will go a long way to helping you identify defects early, write code in a consistent way, and address potential security vulnerabilities as part of development. For example, test-driven development (TDD) requires building test cases first before we write the actual application code. Since we start with test cases, this helps to write the application code in a way that can be easily used to run the test cases. Another example is code formatting. Each of us has our own ways of writing programs and having consistency among application code written by different developers helps to reduce the time that it would otherwise take to adapt to a particular coding style.

Since a single chapter cannot cover such a vast topic in detail, we are going to provide a high-level...