There are challenges involved in distributed neural network training. Some of these challenges include managing different hardware dependencies across master and worker nodes, configuring distributed training to produce good performance, memory benchmarks across the distributed clusters, and more. We discussed some of those concerns in the previous recipes. While keeping such configurations in place, we'll move on to the actual distributed training/evaluation. In this recipe, we will perform the following tasks:
- ETL for DL4J Spark training
- Create a neural network for Spark training
- Perform a test set evaluation