Reading data from files is a very common task that we described in another recipe in this chapter (Writing to and reading from files). However, that's not always the best option. In fact, many Linux/Unix programs can be chained together using a pipe (|) to process an incoming stream. This allows for several things to be done:
- Flexibility on the input source, static text, files, and networking streams—no need to change the programs
- Run several processes, writing only the end result back to disk
- Lazy evaluation of the stream
- Flexible processing up-/downstream (for example, gzipping the output before writing to disk)
If you are not familiar with how this works, the pipe syntax may look cryptic. However, it actually stems from a functional programming paradigm (https://www.geeksforgeeks.org/functional-programming-paradigm/), where pipes...