If you have worked through the two previous recipes, you should already have in place a framework for detecting the presence of a variable number of files in a designated location. You will also have developed a method for processing each of those files as they appear in a prepared list (that is, a table). But consider a situation where there are hundreds or even thousands of files to be processed. Managing all of those files serially would likely prove to be a bottleneck.
In this recipe, we will enhance the file-processing framework by introducing a way to execute the most critical components in parallel.
We will start from where we left off with the previous recipe and to do that, we will build on what we already have: a table that contains a list of files to be processed.