Introducing AWS Step Functions
At re:Invent 2016, AWS announced the Step Functions service as a way to orchestrate common business processes by creating a workflow. A workflow, also referred to as a state machine, is essentially a series of event-driven steps, or States, that denote a single process unit. By chaining these units of work together we are effectively creating an automated process to accomplish an overall goal.
In the case of automating the ML process, we can create a state machine that chains together individual steps to process the training data, train an ML model, evaluate the trained model's performance, and even deploy the model into production.
The advantage of using Step Functions for the ML process, or automating any workflow for that matter, is that we can re-direct the flow based on conditions our outcomes of each step. For example, if a specific step within the workflow fails, we can retry it or redirect the overall flow to follow some alternate...
