Model calibration techniques
There are several ways to calibrate a model. There are two broad categorizations of the calibration techniques based on the nature of the method used to adjust the predicted probabilities to better align with the true probabilities: parametric and non-parametric:
- Parametric methods: These methods assume a specific functional form for the relationship between the predicted probabilities and the true probabilities. They have a set number of parameters that need to be estimated from the data. Once these parameters are estimated, the calibration function is fully specified. Examples include Platt scaling, which assumes a logistic function, and beta calibration, which assumes a beta distribution. We will also discuss temperature scaling and label smoothing.
- Non-parametric methods: These methods do not assume a specific functional form for the calibration function. They are more flexible and can adapt to more complex relationships between the predicted...
