This chapter has the somewhat grandiose and unusual title "The Art of Parallel Programming" as adding the word "art" to the engineering discipline of "programming" may seem odd. While good programming is reflected in good design and good design is often an expression of beauty that exhibits some elemental symmetry—and in the world of the abstract, a recognition of regained inherent simplicity—my intention is to also capture the Harry Potter notion of "Dark Arts": those areas where danger lies. Perhaps an alternate title for this chapter may therefore be "Here, there be dragons!"….
There are many pitfalls that can catch the unwary in the world of parallel programming, and this chapter will alert you to these:
Deadlock – How message passing, in particular, can result in unpredictable program behavior
Numerical instability – The variation in results that can arise when computing in parallel
Random numbers – Ensuring that each processor has its own unique...