Although systems have evolved greatly in terms of technology, communications, and security, data access to source systems is often achieved through flat files. The people responsible for "Operational Systems" (OS) do not like applications that they don't fully understand pulling unknown volumes of data at unknown times from their systems. Even when someone tries to change this, they always get the same reaction: "I'm not letting you alter my system with something I don't know!" They prefer to stay in control and make data sources available in flat files on some 'staging area', in spite of directly opening the OS 'doors'. Therefore, reading text files and subsequent treatment, is a widely used practice mainly in a Data Warehousing scenario known as the PUSH model. In this approach, the OS starts data extraction to an external repository (usually known as a staging area) that will be accessed by an ETL Process.
To handle these file and folder...