Validating Insights Using Statistical Tests

Throughout the journey of EDA, we have collected and noted some interesting patterns for further validation. It is now the right time to test whether whatever we observed previously are actually valid patterns or just appeared to be interesting due to random chance. The most effective and straightforward way to approach this validation is by performing a set of statistical tests and measuring the statistical significance of the pattern. We have a ton of options in the available set of tests to choose from. The options vary based on the type of independent and dependent variable. The following is a handy reference diagram that explains the types of statistical test that we can perform to validate our observed patterns:

Figure 2.24: Validating dependent and independent variables

Let's collect all our interesting patterns into one place here:

• The campaign outcome has a higher chance of yes when the employee variance rate is low.

• The campaign outcome has...