Nuisance parameters and marginalized distributions

While almost any interesting model is multi-parametric, it is also true that not all the parameters we need in order to build a model are of direct interest to us. Sometimes we need to add a parameter just to build the model, even when we do not really care about this parameter. It may happen that we need to estimate the mean value of a Gaussian distribution to answer an important question we have. For such a model, and unless we know the value of the standard deviation, we should also estimate it even if we do not care about it. Parameters necessary to build a model but not interesting by themselves are known as nuisance parameters. Under the Bayesian paradigm, any unknown quantity is treated in the same way, so whether a parameter is or is not a nuisance parameter is more related to our questions than to the parameter itself, the model, or the inference process.

At this point, you may think that having to build a model with parameters...