Dealing with unbalanced datasets

At this point, you might have realized why data preparation is probably the longest part of the data scientist’s work. You have learned about data transformation, missing data values, and outliers, but the list of problems goes on. Don’t worry – you are on the right journey to master this topic!

Another well-known problem with ML models, specifically with classification problems, is unbalanced classes. In a classification model, you can say that a dataset is unbalanced when most of its observations belong to one (or some) of the classes (target variable).

This is very common in fraud identification systems: for example, where most of the events belong to a regular operation, while a very small number of events belong to a fraudulent operation. In this case, you can also say that fraud is a rare event.

There is no strong rule for defining whether a dataset is unbalanced or not, it really depends on the context of your business...