Data analysts and scientists spend most of their time cleaning data and pre-processing messy datasets. While this activity is less talked about, it is one of the most performed activities and one of the most important skills for any data professional. Mastering the skill of data cleaning is necessary for any aspiring data scientist. Data cleaning and pre-processing is the process of identifying, updating, and removing corrupt or incorrect data. Cleaning and pre-processing results in high-quality data for robust and error-free analysis. Quality data can beat complex algorithms and outperform simple and less complex algorithms. In this context, high quality means accurate, complete, and consistent data. Data cleaning is a set of activities such as handling missing values, removing outliers, feature encoding, scaling, transformation, and splitting.
This chapter...
