Understanding the concepts of parameter estimation and the features of estimators

A introduction to estimation theory requires a good mathematical understanding and careful derivation. Here, I am going to use layman's terms to give you a brief but adequate introduction so that we can move on to concrete examples quickly.

Estimation, in statistics, refers to the process of estimating unknown values from empirical data that involves random components. Sometimes, people confuse estimation with prediction. Estimation usually deals with hidden parameters that are embodied in a known dataset: things that already happened, while prediction tries to predict values that are explicitly not in the dataset: things that haven't happened. For example, estimating the population of the world 1,000 years ago is an estimation problem. You can use various kinds of data that may contain information about the population. The population is a number that will not change but is unknown. On the...