Chapter 2.  Distributed Deep Learning for Large-Scale Data

	"In God we trust, all others must bring data"
	--W. Edwards Deming

In this exponentially growing digital world, big data and deep learning are the two hottest technical trends. Deep learning and big data are two interrelated topics in the world of data science, and in terms of technological growth, both are critically interconnected and equally significant.

Digital data and cloud storage follow a generic law, termed as Moore's law [50], which roughly states that the world's data are doubling every two years; however, the cost of storing that data decreases at approximately the same rate. This profusion of data generates more features and verities, hence, to extract all the valuable information out of it, better deep learning models should be built.

This voluminous availability of data helps to bring huge opportunities for multiple sectors. Moreover, big data, with its analytic part...