Outliers
In this chapter, we want to deal with the manipulation of big data sources to address data outliers. So let's have a quick reminder for the reader:
Outliers can be defined as:
A data point that is way out of keeping with the others
That piece of data that doesn't fit
Either a very high value or a very low value
Unusual observations within the data
An observation point that is distant from all others
Options for outliers
The options that are generally accepted for dealing with found outliers in big data are:
Delete: This includes the outlier values or even the actual variable where the outliers exist
Transform: This includes the values or the variable itself
Delete
If you have just a few outliers, you may decide to simply delete those outlying values (they then become blank or missing values, which usually are easier to deal with in a visualization). Also, if the variable just doesn't make sense, or if there are just too many outliers in that variable (or maybe you just don't need the variable...
