Null Hypothesis Significance Testing

For better or worse, Null Hypothesis Significance Testing (NHST) is the most popular hypothesis testing framework in modern use. So, even though there are competing approaches that—at least in some cases—are better, you need to know this stuff up and down!

Okay—Null Hypothesis Significance Testing—those are a bunch of big words. What do they mean?

NHST is a lot like being a prosecutor in the United States' or Great Britain's justice system. In these two countries—and a few others—the person being charged is presumed innocent, and the burden of proving the defendant's guilt is placed on the prosecutor. The prosecutor then has to argue that the evidence is inconsistent with the defendant being innocent. Only after it is shown that the extant evidence is unlikely if the person is innocent, does the court rule a guilty verdict. If the extant evidence is weak, or is likely to be observed even if the dependent is innocent, then the court rules not guilty. That...