Checking unsanitized data

Very often, there will be errors or mistakes in data that can severely complicate analysis—especially with public data or data outside of your organization. We will refer to this data as unsanitized data in this chapter. For example, say there is a stray comma or punctuation mark in a column that was supposed to be numeric. If we aren't careful, R will read this column as character , and subsequent analysis may, in the best case scenario, fail. It is also possible, however, that our analysis will silently chug along and return an unexpected result. This will happen, for example, if we try to perform linear regression using the punctuation-containing-but-otherwise-numeric column as a predictor, which will compel R to convert it into a factor, thinking that it is a categorical variable.

In the worst-case scenario, an analysis with unsanitized data may not error out or return nonsensical results, but return results that look plausible but are actually incorrect. For...