The first thing you will likely need to do is partition data in files into rows of data and columns of data. We saw some transformations in the previous chapters that allow us to manipulate data one row at a time. For this chapter, we'll assume the rows of your data correspond with the lines of data in your files. If this isn't the case, this may be the first thing you want to do in your pipeline.
Given that we have some rows of data in our file or stream, we would like to view those rows in a columnar fashion, such as a traditional database. We can do this using the help of the cut command. cut will allow us to chop the lines of the file into columns by a delimiter, and to select which of those columns get passed through to the output.
If your data is a comma-separated or tab-separated file, cut is quite simple:
zcat amazon_reviews_us_Digital_Ebook_Purchase_v1_01...