Introducing schema mapping
As you know, before we can ingest any data into our ADX instance, we need to create tables in our database to store the data. Similar to a SQL database, ADX tables are two-dimensional, meaning they consist of rows and columns. When we create tables, we need to declare the column name and data type. A data type refers to the type of data a column can store, such as strings, dates, and numbers. We will create our own tables later in the chapter.
How do we ensure the data we are ingesting is imported into the correct tables and rows? The destination table is specified during the data connection creation and the columns are mapped to the incoming data using schema maps.
As you will see in the section Ingesting data from Blob storage using Azure Event Grid, it is possible for the source file to contain more columns than you are interested in. We will take a data source with over 60 columns and create a schema map to ingest only the columns we are interested...