In this chapter, we spent time cleaning the data we acquired in Chapter 6, First Script – Geocoding with Web APIs. Unless data was carefully prepared for the exact purpose of analysis, the chances are that cleaning will take a lot of time and effort. Here, we learned the basics of pandas, and how to filter and mask the data. We discussed how to investigate missing values, saw how to use regular expressions to extract specific values from non-structured text, creating data of a proper structure and type, and learned how to apply custom functions to each cell in the entire Series or DataFrame and then used that information to geocode locations where we lacked coordinates.
Finally, we stored all the data we processed, along with the original values, in another CSV file, ready to be explored in our next chapter.
