Understanding data types, formats, and encodings
In this section, you will learn about the various data types and data formats. We will also cover compression and how compression and formats go together. After that, we will briefly discuss data encodings. This section will prepare you to understand these basic features of data, which will be of use when we discuss data storage and databases in the upcoming sections.
Data types
All datasets that are used in modern-day data engineering can be broadly classified into one of three categories, as follows:
- Structured data: This is a type of dataset that can easily be mapped to a predefined structure or schema. It usually refers to the relational data model, where each data element can be mapped to a predefined field. In a structured dataset, usually, the number of fields, their data type, and the order of the fields are well defined. The most common example of this is a relational data structure where we model the data structure...