Book Image

Mastering Parallel Programming with R

By : Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup
Book Image

Mastering Parallel Programming with R

By: Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup

Overview of this book

R is one of the most popular programming languages used in data science. Applying R to big data and complex analytic tasks requires the harnessing of scalable compute resources. Mastering Parallel Programming with R presents a comprehensive and practical treatise on how to build highly scalable and efficient algorithms in R. It will teach you a variety of parallelization techniques, from simple use of R’s built-in parallel package versions of lapply(), to high-level AWS cloud-based Hadoop and Apache Spark frameworks. It will also teach you low level scalable parallel programming using RMPI and pbdMPI for message passing, applicable to clusters and supercomputers, and how to exploit thousand-fold simple processor GPUs through ROpenCL. By the end of the book, you will understand the factors that influence parallel efficiency, including assessing code performance and implementing load balancing; pitfalls to avoid, including deadlock and numerical instability issues; how to structure your code and data for the most appropriate type of parallelism for your problem domain; and how to extract the maximum performance from your R code running on a variety of computer systems.
Table of Contents (13 chapters)

Building an MPI R package – SPRINT


Now that we have built an R shared object library that contains MPI code which is callable from R, let's investigate how to create an R package that contains a number of MPI-enabled functions, each callable from R.

Building a package for this can be useful for various reasons, including the following:

  • Maintainability: If each function has its own MPI setup and teardown, then you could end up with a lot of duplicate code to maintain.

  • Flexibility: From one invocation of MPI in your R script, you can easily call multiple, different MPI-enabled functions according to your needs.

  • Efficiency: If each function has its own separate shared object library, then each will go through their own MPI_Init/MPI_Finalize stages when called, thus adding to the runtime.

The SPRINT package provides an R user with just such a suite of parallelized functions callable from R that exploit MPI. In the following sections, we will show you how to add your own function to the SPRINT package...