One of the questions with device change is why so much improvement is observed when the device is switched from CPU to GPU. As the deep learning architecture involves a lot of matrix computations, GPUs help expedite these computations using a lot of parallel cores, which are usually used for image rendering.
The power of GPU has been utilized by a lot of algorithms to accelerate the execution. The following recipe provides some benchmarks of matrix computation using the gpuR package. The gpuR package is a general-purpose package for GPU computing in R.