Using log-level types
Now that we have been introduced to how and where to insert logs, let’s understand log types or levels. Each log level has its own degree of relevance inside any system. For instance, the console output does not show debug messages by default.
We already covered how to log levels using PySpark in the Inserting formatted SparkSession logs to facilitate your work recipe in Chapter 6. Now we will do the same using only Python. This recipe aims to show how to set logging levels at the beginning of your script and insert the different levels inside your code to create a hierarchy of priority for your logs. With this, you can create a structured script that allows you or your team to monitor and identify errors.
Getting ready
We will use only Python code. Make sure you have Python version 3.7 or above. You can use the following command on your CLI to check your version:
$ python3 –-version Python 3.8.10
The following code execution can be...