The MapReduce pattern was introduced in order to handle big data problems such as large-scale computing requirements spanning across a cluster of servers. The pattern can also be used on single-core machines.
A MapReduce program is composed of two tasks: map and reduce. The input for the MapReduce program is passed as a set of key-value pairs and the output is also received as such.
To implement this pattern, we need to start by writing a map function that takes in data (key/value pair) as a single input value and converts it into another set of intermediate data (key/value pair). The user then writes a reduce function that takes the output from the map function (key/value pair) as input and combines the data with a smaller dataset containing any number of rows of data.
Let's look at how to implement a basic MapReduce pattern using LINQ and convert it...