Trimming outliers from the dataset

Trimming, or truncating, is the process of removing observations that show outliers in one or more variables in the dataset. There are three commonly used methods to set the boundaries beyond which a value can be considered an outlier. If the variable is normally distributed, the boundaries are given by the mean plus or minus three times the standard deviation, as approximately 99% of the data will be distributed between those limits. For normally, as well as not normally, distributed variables, we can determine the limits using the inter-quartile range proximity rules or by directly setting the limits to the 5^th and 95^th quantiles. We covered the formula for the inter-quartile range proximity rule in the Getting ready section of the Highlighting outliers recipe in Chapter 1, Foreseeing Variable Problems...