Book Image

Mastering Parallel Programming with R

By : Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup
Book Image

Mastering Parallel Programming with R

By: Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup

Overview of this book

R is one of the most popular programming languages used in data science. Applying R to big data and complex analytic tasks requires the harnessing of scalable compute resources. Mastering Parallel Programming with R presents a comprehensive and practical treatise on how to build highly scalable and efficient algorithms in R. It will teach you a variety of parallelization techniques, from simple use of R’s built-in parallel package versions of lapply(), to high-level AWS cloud-based Hadoop and Apache Spark frameworks. It will also teach you low level scalable parallel programming using RMPI and pbdMPI for message passing, applicable to clusters and supercomputers, and how to exploit thousand-fold simple processor GPUs through ROpenCL. By the end of the book, you will understand the factors that influence parallel efficiency, including assessing code performance and implementing load balancing; pitfalls to avoid, including deadlock and numerical instability issues; how to structure your code and data for the most appropriate type of parallelism for your problem domain; and how to extract the maximum performance from your R code running on a variety of computer systems.
Table of Contents (13 chapters)

Adding a new function to the SPRINT package


Let's now add our own function to the SPRINT package. This new function will be called phello(). We will use our earlier MPI Hello World example as the basis for this. This will involve the following tasks:

  • Downloading the SPRINT source code.

  • Creating the R stub file: This enables the desired functionality to be callable from R on the Master process. It calls the interface function for this functionality.

  • Adding the interface function: The interface function is the C equivalent of the R stub. It is also executed on the Master process. It is responsible for broadcasting the command code for the implementation function that the Worker processes are to execute.

  • Adding the implementation function: Each command code has a corresponding implementation function. On receipt of the command code, this function is executed on the Worker processes. Additionally, it is also executed on the Master process.

  • Connecting the stub and functions: Update the relevant SPRINT...