It might seem worthless to stop and think about what type of data we have before getting into the fun stuff, such as statistics and machine learning, but this is arguably one of the most important steps you need to take to perform data science.

The same principle applies to data science. When given a dataset, it is tempting to jump right into exploring, applying statistical models, and researching the applications of machine learning in order to get results faster. However, if you don't understand the type of data that you are working with, then you might waste a lot of time applying models that are known to be ineffective with that specific type of data.

When given a new dataset, I always recommend taking about an hour (usually less) to make the distinctions mentioned in the following sections.

The distinction between structured and unstructured data is usually the first question you want to ask yourself about the entire dataset. The answer to this question can mean the difference between needing three days or three weeks of time to perform a proper analysis.

The basic breakdown is as follows (this is a rehashed definition of organized and unorganized data in the first chapter):

Here are a few examples that could help you differentiate between the two:

When you ask a data scientist, "what type of data is this?", they will usually assume that you are asking them whether or not it is mostly quantitative or qualitative. It is likely the most common way of describing the specific characteristics of a dataset.

For the most part, when talking about quantitative data, you are usually (not always) talking about a structured dataset with a strict row/column structure (because we don't assume unstructured data even has any characteristics). All the more reason why the pre-processing step is so important.

These two data types can be defined as follows:

So far in this chapter, we have looked at the differences between structured and unstructured data, as well as between qualitative and quantitative characteristics. These two simple distinctions can have drastic effects on the analysis that is performed. Allow me to summarize before moving on the second half of the chapter.

Data as a whole can either be structured or unstructured, meaning that the data can either take on an organized row/column structure with distinct features that describe each row of the dataset, or exist in a free-form state that usually must be pre-processed into a form that is easily digestible.

If data is structured, we can look at each column (feature) of the dataset as being either quantitative or qualitative. Basically, can the column be described using mathematics and numbers, or not? The next part of this chapter will break down data into four very specific and detailed levels. At each order, we will apply more complicated rules of mathematics...

It is generally understood that a specific characteristic (feature/column) of structured data can be broken down into one of four levels of data. The levels are as follows:

As we move down the list, we gain more structure and, therefore, more returns from our analysis. Each level comes with its own accepted practice in measuring the center of the data. We usually think of the mean/average as being an acceptable form of a center. However, this is only true for a specific type of data.

It is possible to impose structure on data. For example, while I said that you technically cannot use a mean for the one to five data at the ordinal scale, many statisticians would not have a problem using this number as a descriptor of the dataset.

The level at which you are interpreting data is a huge assumption that should be made at the beginning of any analysis. If you are looking at data that is generally thought of at the ordinal level and applying tools such as the arithmetic mean and standard deviation, this is something that data scientists must be aware of. This is mainly because if you continue to hold these assumptions as valid in your analysis, you may encounter problems. For example, if you also assume divisibility at the ordinal level by mistake, you are imposing a structure where the structure may not exist.

The type of data that you are working with is a very large piece of data science. It must precede most of your analysis because the type of data you have impacts the type of analysis that is even possible!

Whenever you are faced with a new dataset, the first three questions you should ask about it are the following:

The answers to these questions will not only impact your knowledge of the data at the end but will also dictate the next steps of your analysis. They will dictate the types of graphs you are able to use and how you interpret them in your upcoming data models. Sometimes, we will have to convert from one level to another in order...