Speeding up queries using cache policies
As you have seen in Chapter 1, Introducing Azure Synapse Data Explorer, Data Explorer pools can manage very large amounts of data. They separate the compute layer from the storage layer, allowing you to scale massively in storage, regardless of how much compute you have allocated in your Data Explorer pool.
When dealing with large volumes of data, it’s useful to understand what data you need readily available as needed, and what data can be stored as an archive, meaning that it is still available but maybe at a cheaper location that is slower to retrieve. This is the concept of hot data and cold data. Data accessed frequently is designated as hot data and should be quick to retrieve. Data that is less frequently accessed but is still needed is designated as cold data, and can typically be stored in cheaper storage that is still reliable but slower to retrieve. The implication here is not only on performance but also on cost: cold storage...