Book Image

Mastering Parallel Programming with R

By : Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup
Book Image

Mastering Parallel Programming with R

By: Simon R. Chapple, Terence Sloan, Thorsten Forster, Eilidh Troup

Overview of this book

R is one of the most popular programming languages used in data science. Applying R to big data and complex analytic tasks requires the harnessing of scalable compute resources. Mastering Parallel Programming with R presents a comprehensive and practical treatise on how to build highly scalable and efficient algorithms in R. It will teach you a variety of parallelization techniques, from simple use of R’s built-in parallel package versions of lapply(), to high-level AWS cloud-based Hadoop and Apache Spark frameworks. It will also teach you low level scalable parallel programming using RMPI and pbdMPI for message passing, applicable to clusters and supercomputers, and how to exploit thousand-fold simple processor GPUs through ROpenCL. By the end of the book, you will understand the factors that influence parallel efficiency, including assessing code performance and implementing load balancing; pitfalls to avoid, including deadlock and numerical instability issues; how to structure your code and data for the most appropriate type of parallelism for your problem domain; and how to extract the maximum performance from your R code running on a variety of computer systems.
Table of Contents (13 chapters)

Genomics with a supercomputer


Now that you have some knowledge about Genomics, let's look at how a supercomputer can help an R user investigating bacterial infection in newborn babies.

The goal

It is possible to use genomic data (like microarray gene expression data) to identify sets of genes that, taken together, can predict if a new biological sample belongs to a particular class sample (that is, a healthy sample or a diseased sample). In the case study presented here, we will look at the research by the Division of Infection and Pathway Medicine at The University of Edinburgh into diagnosing bacterial infection in young infants by measuring gene expression in blood samples. We want to look at how effectively a supercomputer can be used by R to process the large gene expression datasets involved.

The ARCHER supercomputer

The supercomputer used is Cray XC30 MPP. This forms part of ARCHER, the UK's academic national supercomputing service. At the time of writing, (March 2015), this service consists...