While probability allows us to measure and calculate the odds of events or outcomes occurring, statistics allows us to make judgments and decisions given data generated by some unknown probability model. We use the data to learn the properties of the underlying probabilistic model. We call this process parametric inference.
Essential concepts in statistics
Estimation
In estimation, our objective is given n iid samples with the same distribution as X (the probability model). If the PDF and probability mass function (PMF) is 
, we need to find θ.
Formally, we define a statistic as an estimate of θ.
A statistic is a function, T, of the data, 
, so that our estimate is 
. Therefore, T(x) is the sampling distribution...
