Training models at scale
In an earlier section of this chapter, we listed and studied what the industry experts agree on as the most common phases of any predictive analytics project.
To recall, they are as follows:
- Defining the data source
- Profiling and preparation of the data source
- Determining the question(s) that you want to ask your data
- Choosing an algorithm to train on the data source
- Application of a predictive model
In a predictive analytics project using big data, those same phases are present, but may be slightly varied and require some supplementary efforts.
Pain by phase
In the initial phase of a project, once you've chosen a source for your data (determined the data source), the data must be attained. Some industry experts describe this as the acquisition and recording of data. In a predictive project that involves a more common data source, access to the data might be as straightforward as opening a file on your local disk; with a big data source, it's a bit more difficult...
