Identifying and tackling multicollinearity

Multicollinearity is a situation where one (or more) of independent variables can be expressed as a linear combination of some other independent variables.

For example, consider a situation where we try to predict the power consumption for a state using population, number of households, and number of power plants located in the state. In a situation like this, one might clearly deduce that the more people living in the state, the higher number of households one might expect, that is, the number of households can be represented by some (close to) linear relationship of the state's population.

Now, if we were to estimate a model based on a data that is collinear, very good chances are that one (or even all the variables that are collinear) will turn out as insignificant. In contrast, removing the collinear variables (and keeping only the variable that is the most correlated with our dependent variable, that is, explains most of its variation) will not...