In this chapter, we leverage databases in our scripts to help us accomplish meaningful tasks when working with large quantities of data. We have chosen a simple example that demonstrates the capabilities and benefits of a database backend with Python. We will store file metadata that has been recursively indexed from a given root directory into a database and then query it to generate reports. Although it may seem a simple feat, the purpose is to focus on showcasing multiple ways in which we can interact with a database in Python by creating an active file listing.
In this chapter, we will delve into the following areas:
The basic design and implementation of SQLite3 databases
Working with these databases in Python using the first and third-party modules
Understanding how to recursively iterate through directories in Python
The knowledge of file system metadata and methods of accessing it using Python
Crafting CSV and HTML reports for the review process