We will now see how to program our own point-wise (or equivalently, element-wise) operations directly onto our GPU with the help of PyCUDA's ElementWiseKernel function. This is where our prior knowledge of C/C++ programming will become useful—we'll have to write a little bit of inline code in CUDA C, which is compiled externally by NVIDIA's nvcc compiler and then launched at runtime by our code via PyCUDA.
We use the term kernel quite a bit in this text; by kernel, we always mean a function that is launched directly onto the GPU by CUDA. We will use several functions from PyCUDA that generate templates and design patterns for different types of kernels, easing our transition into GPU programming.
Let's dive right in; we're going to start by explicitly rewriting the code...
