Parallel programming (sometimes referred to as concurrent programming) refers to executing several processes at the same time. By processes, I mean either operating level processes, threads, or maybe programs on different computers connected in a network. By itself, that does not sound interesting. Simply running several programs at the same time is useful, but it's hard to see what is there to talk about in this book or how it is related to scientific computing. You run them, you get the results, maybe process those results, and so on. Any kinds of complication (and at the same time interesting stuff) only happen when those processes have to interact with each other.
In fact, many algorithms can be written as several processes working together toward a unified goal and exchanging information in order to accomplish that goal. This is particularly attractive at this time when CPUs with multiple cores are widely available, and the clock frequencies of...