Scaling and cost management
One of the aspects that we have not discussed in detail so far is scaling. One of the design principles of cloud computing is elasticity, and Azure allows us to scale our resources on demand. Scaling in the context of elasticity comes in two dimensions. One is vertical scaling, which refers to increasing the specification of a VM or ADX engine node. For instance, changing the engine SKU from Standard_E64i_v3
to Standard_E80ids_v4
is an example of vertical scaling.
The second dimension is horizontal scaling. Horizontal scaling refers to adding more VMs or engines. For instance, increasing the number of engines is a form of horizontal scaling.
ADX can take care of scaling for us and this is referred to as auto-scaling, but there can be up to 10 minutes of downtime. If downtime is an issue, then you can also use manual scaling and decide when to scale your cluster—for instance, you could manually scale your cluster outside of peak hours.
If...