Scalding provides a rich set of core operations to perform data transformations. Map-like operations apply a function to each tuple in the pipe. Join operations can join data from multiple pipes. Pipe operations allow us to concatenate or debug pipes. Grouping/Reducing operations group related data together. Also, for data that has been grouped, there is a rich set of group operations.
These operations are internally translated into map phases of MapReduce and apply a function to every row of data. The syntax of the map operation is:
pipe.map(existingFields -> additionalFields) { function }The map operation uses some of the existing fields of a pipe as input and creates a pipe with additional fields by applying a function to the elements of the input. In the following example, a new field 'priceWithVAT is introduced:
pipe.map('price -> 'priceWithVAT) { price: Double => price*1.20 }Operations can be executed to multiple...