With the advent of big data and multi-core machines, it becomes apparent that in order to keep up with the amount of data, we need to begin scaling our systems out.
Although making use of multiple cores simultaneously (that is, in parallel) is effectively considered scaling up (or vertical scalability), some of the principles we can use to reliably do this can be applied to scaling out (or horizontal scalability).
One of the difficult parts of multithreaded programming is the synchronization of shared data. Data that can be read and written in two different places at the same time needs to make sure those operations don't overlap across threads and corrupt data. This is the act of making something Thread Safe or writing code for thread synchronization. This process is tricky and sometimes hard to get right. More detail on this can be found in Chapter 8,Parallelization Practices.
Another difficult part of multithreaded programming and distributed design is the separation...