Characteristics of big data
For you to determine if your data source qualifies as big data or as needing special handling, you can start by examining your data source in the following areas:
- The volume (amount) of data.
- The variety of data.
- The number of different sources and spans of the data.
Let's examine each of these areas.
Volume
If you are talking about the number of rows or records, then most likely your data source is not a big data source since big data is typically measured in gigabytes, terabytes, and petabytes. However, space doesn't always mean big, as these size measurements can vary greatly in terms of both volume and functionality. Additionally, data sources of several million records may qualify as big data, given their structure (or lack of structure).
Varieties
Data used in predictive models may be structured or unstructured (or both) and include transactions from databases, survey results, website logs, application messages, and so on (by using a data source consisting...