In order to understand scheduling at the level of the application developer (and how you can leverage this knowledge in actual code), we must first cover some required background information.
The first and very important concept for the developer to understand is that OSes maintain a construct called the Kernel Schedulable Entity (KSE). The KSE is the granularity at which the OS scheduling code operates. In effect, what object exactly does the OS schedule? Is it the application, the process, the thread? Well, the short answer is that the KSE on the Linux OS is a thread. In other words, all runnable threads compete for the CPU resource; the kernel scheduler is ultimately the arbiter that decides which thread gets which CPU core and when.
Next, we present an overview of the process, or thread's, state machine.