So far we have been discussing hardware and the number of CPUs, hardware cores, and logical cores; now, let's transition to software threads and the OS. A software application or service can run in one or many processes and threads. Typically, a software application has a user interface and is run by a user of the computer, while a software process is run by the OS and runs in the background. Both of these are types of software that are being executed by the computer they are running on.

Each application or service in turn has one or several processes that they actually execute inside. Processes are the running objects of an application or service. Also, each process has one or many execution threads (or software threads). The threads are the items that the scheduler schedules on the cores. I know this might seem confusing but it is important to understand the hardware from the ground up and how many physical cores can execute a software instruction, and the software from the top-down to each software thread that executes on a core.

The scheduler for an operating system is the subsystem of the OS that manages all of the software threads currently running and allocates execution time on the cores (both physical and logical) of the computer. Execution time is divided up in machine cycles, and a machine cycle is a tick of the computer's clock.

The scheduler determines which software threads run on which core (physical) each clock cycle. So, during each clock cycle in a computer, each core can execute an instruction of a software thread. Remember that using hyperthreading technology, the scheduler treats each logical core as a physical core. But in actuality, in each clock cycle, each physical core executes a single software thread's instruction.

Also important in our parallel development and estimation of performance gains is that we are assuming in our estimates that all hardware cores are available to our software application each clock cycle. In reality, most computers have many processes running at a given time and utilize some of the execution time of a core.

In Windows, the Task Manager provides some useful information to see what is running and consuming hardware resources. We have already looked at the Performance tab. Now, let's look at the Processes tab:

This tab in the Task Manager shows various information about the processes running on a Windows computer. As you can see from the Threads column in the preceding screenshot, some processes have many threads of execution at any given time.

You can go to the View menu and select Select Columns to change which columns of information are displayed by the Task Manager for each process.