Detecting and removing outliers

Outliers are usually dangerous values for data science activities, since they produce heavy distortions within models and algorithms.

Their detection and exclusion is, therefore, a really crucial task.

This recipe will show you how to easily perform this task.

We will compute the I and IV quartiles of a given population and detect values that far from these fixed limits.

You should note that this recipe is feasible only for univariate quantitative population, while different kind of data will require you to use other outlier-detection methods.

How to do it...

1. Compute the quantiles using the quantile() function:

   quantiles <- quantile(tidy_gdp_complete$gdp, probs = c(.25, .75))
   

2. Compute the range value using the IQR() function:

   range <- 1.5 * IQR(tidy_gdp_complete$gdp)
   

3. Subset the original data by excluding the outliers:

   normal_gdp <- subset(tidy_gdp_complete,
   tidy_gdp_complete$gdp > (quantiles[1] - range) & tidy_gdp_complete$gdp < (quantiles[2] + range...