Book Image

Mastering Parallel Programming with R

By : Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup
Book Image

Mastering Parallel Programming with R

By: Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup

Overview of this book

R is one of the most popular programming languages used in data science. Applying R to big data and complex analytic tasks requires the harnessing of scalable compute resources. Mastering Parallel Programming with R presents a comprehensive and practical treatise on how to build highly scalable and efficient algorithms in R. It will teach you a variety of parallelization techniques, from simple use of R’s built-in parallel package versions of lapply(), to high-level AWS cloud-based Hadoop and Apache Spark frameworks. It will also teach you low level scalable parallel programming using RMPI and pbdMPI for message passing, applicable to clusters and supercomputers, and how to exploit thousand-fold simple processor GPUs through ROpenCL. By the end of the book, you will understand the factors that influence parallel efficiency, including assessing code performance and implementing load balancing; pitfalls to avoid, including deadlock and numerical instability issues; how to structure your code and data for the most appropriate type of parallelism for your problem domain; and how to extract the maximum performance from your R code running on a variety of computer systems.
Table of Contents (13 chapters)

Chapter 6. The Art of Parallel Programming

This chapter has the somewhat grandiose and unusual title "The Art of Parallel Programming" as adding the word "art" to the engineering discipline of "programming" may seem odd. While good programming is reflected in good design and good design is often an expression of beauty that exhibits some elemental symmetry—and in the world of the abstract, a recognition of regained inherent simplicity—my intention is to also capture the Harry Potter notion of "Dark Arts": those areas where danger lies. Perhaps an alternate title for this chapter may therefore be "Here, there be dragons!"….

There are many pitfalls that can catch the unwary in the world of parallel programming, and this chapter will alert you to these:

  • Deadlock – How message passing, in particular, can result in unpredictable program behavior

  • Numerical instability – The variation in results that can arise when computing in parallel

  • Random numbers – Ensuring that each processor has its own unique...