Confidence intervals
Our explanation of hypothesis testing in the previous section focused a lot on how to calculate the p-value. We explained that this focus on p-values can be dangerous, as well as that reporting results from a hypothesis test should include not just the p-value but also the estimate of the effect (e.g., the estimate of for our difference in means example). The only problem is that our estimate is based on sample data, so it has an inherent uncertainty associated with it. Ideally, we should also report the degree of uncertainty associated with our estimate.
To summarize our estimate of the effect and also the uncertainty of that estimate, we’ll introduce a new concept, a confidence interval. To explain what a confidence interval is, we’ll first recap some results from the previous section.
Suppose we have run the two-sample z-test hypothesis test of the previous section. We used the calculated value of to reject the null hypothesis that , so...