The execution of code on a GPU is divided among streaming multiprocessors, blocks, and threads. The GPU has several different memory spaces, with each having particular features and uses and different speeds and scopes. This memory space is hierarchically divided into different chunks, like global memory, shared memory, local memory, constant memory, and texture memory, and each of them can be accessed from different points in the program. This memory architecture is shown in preceding diagram:
As shown in the diagram, each thread has its own local memory and a register file. Unlike processors, GPU cores have lots of registers to store local data. When the data of a thread does not fit in the register file, the local memory is used. Both of them are unique to each thread. The register file is the fastest memory. Threads in the same blocks have shared memory...
