Summary

In this chapter, we deep-dived into A/B testing and experimentation. We reviewed what the purpose of experimentation is, the different types of experiments, and the different types of errors that can happen. We delved deep into what a p-value is and is not and how to interpret it. We review the math on calculating power and false positive risk, to measure the risks on our experiments. We also reviewed the common pitfalls of experimentation and how to avoid them.

Equipped with the necessary knowledge, you can plan and conduct reliable experiments, recognize and address potential issues from poor design, select alternative study types when RCTs are impractical, and implement your analysis in Python.

We have now reached the end of our journey into marketing analytics with Python. From here, you should have a sense and understanding of the basics of the most common methods currently available to you. This book is not, however, exhaustive. As mentioned in the Preface, each...