A path forward

So, the inkling of having more than enough data for training a model seems very appealing.

Big data sources would appear to answer this desire, however in practice, a big data source is not often (if ever) analyzed in its entirety. You can pretty much count on performing a sweeping filtering process aimed to reduce the big data into small(er) data (more on this in the next section).

In the following section, we will review various approaches to addressing the various challenges of using big data as a source for your predictive analytics project.

Opportunities

In this section, we offer a few recommendations for handling big data sources in predictive analytic projects using R. Also, we'll offer some practical use case examples.

Bigger data, bigger hardware

We are starting with the most obvious option first.

To be clear, R keeps all of its objects in memory, which is a limitation if the data source gets too large. One of the easiest ways to deal with big data in R is simply to...