When operating distributed clusters of servers, we need to figure out a good approach to meet the need to communicate with the servers and the services they offer. Imagine a cluster of hundreds, thousands, or even tens of thousands of servers running on an environment such the spot service, across several regions and availability zones. Now, imagine having to determine the following at any given time:
- The network address or DNS name of each server
- The current load on the server
- Which server is available to take a request and process it
This would be a daunting task with a stable environment, let alone a spot environment where the servers can pop in and out of existence on the EC2 compute layer.
So, instead of trying to figure out which server I should send my request to, we can simply let the servers themselves decide when there are enough resources to...