Since running a MapReduce program will require a considerable amount of time, varying from a few minutes to several hours, testing and debugging become very important. In this recipe, we will illustrate some techniques you can use to troubleshoot an R MapReduce program.
In this recipe, you should have completed the previous recipe by installing rmr2
into an R environment.
Perform the following steps to test and debug an R MapReduce program:
- First, you can configure the backend as local in
rmr.options
:
> rmr.options(backend = 'local')
- Again, you can execute the number squared MapReduce program mentioned in the previous recipe:
> b.time = proc.time() > small.ints= to.dfs(1:100000) > result = mapreduce(input = small.ints, map = function(k,v) cbind(v,v^2)) > proc.time() - b.time
- In addition to this, if you want to print the structure information of any variable in the MapReduce program, you can use the
rmr.str...