Queue effects
An important element of asynchronous tasks is the effect that introducing a queue may have. As we've seen, the background tasks are slow, meaning that any worker running them will be busy for some time.
Meanwhile, more tasks can be introduced, which may mean that the queue starts building up.
Figure 7.4: Single queue
On the one hand, this can be a capacity problem. If the number of workers is not sufficient to handle the average number of tasks introduced in the queue, the queue will build up until it reaches its limit, and new tasks will be rejected.
But typically, the load doesn't work like a constant influx of tasks. Instead, there are times when there are no tasks to execute, and other times when there's a sudden spike in the number of tasks to be executed, filling the queue. Also, there's a need to calculate the right number of workers to keep running to be sure that the waiting period for those spikes, where a task gets...