There are mainly two aspects to consider:
- Training time
- Scoring time
Both will act as constraints as you are developing your pipelines.
Let's think about the limitations that training and scoring time bring to the table. Requirements for training time will usually determine the algorithms that you will include in your candidate list. For example, logistic regression and Support Vector Machines (SVMs) are fast-to-train algorithms, and this might be important to you, especially if you are prototyping ideas quickly using big data. They are also fast when it comes to scoring. There are different implementations for both, and also different options are available for solvers, which make these two convenient for many ML use cases.
However, for something like a deep neural network, training and scoring time are very limiting constraints as you may...
