Building a logical folder structure
We learned about efficient folder structures in Chapter 2, Designing a Data Storage Structure, where we explored the best practices for storing data for batch processing and streaming scenarios. The rule of thumb is to store the data in a hierarchical date folder structure, with the date part added toward the end, as shown here:
{Region}/{SubjectMatter(s)}/{yyyy}/{mm}/{dd}/{hh}/
We can have more intermediate folders in the folder path based on our business requirements. Please refer to Chapter 2, Designing a Data Storage Structure, to refresh your memory on designing efficient folder structures.
To create a container on Azure Data Lake Gen 2, you can use the following command via the Azure CLI:
az storage fs create -n <container name> --account-name <account name> --auth-mode login
Once you have created a container, you can easily create folders in it by using the following command:
az storage fs directory create -n...