Hyper-threading versus physical CPU cores
Hyper-threading is a technology that offers extra virtual CPU cores to your physical cores. The idea is that, because these virtual CPU cores have separate caches and other resources, you can more efficiently switch between multiple tasks. If you task-switch between two heavy processes, the CPU won’t have to unload/reload all caches. When it comes to actual CPU instruction processing, however, it will not help you.
When you truly maximize CPU usage, it is generally better to only use the physical processor count. To demonstrate how this affects the performance, we will run a simple test with several process counts. Since my processor has 8 cores (16 if you include hyper-threading), we will run it with 1, 2, 4, 8, 16, and 32 processes to demonstrate how it affects the performance:
import timeit
import multiprocessing
def busy_wait(n):
while n > 0:
n -= 1
def benchmark(n, processes, tasks):
with multiprocessing...