What happens if the entire datacenter goes down? To be prepared for this eventuality, you need to run your application cluster in more than one datacenter, and ensure that both deployments are in sync in terms of data. Building such architectures is typically under the purview of business continuity planning (BCP) and disaster recovery (DR).
A common way to have DNS switch between deployments in two datacenters. A DNS name, such as www.mysite.com, resolves to a VIP of 4.4.4.4 with a specific time-to-live (TTL). This layer can be made intelligent and, in the case of a datacenter outage, repoint the DNS name to a backup VIP, say 5.5.5.5. For doing this we need the deployments to happen in both datacenters and that the data is replicated (usually asynchronously) between the deployments. This scheme is described in the following diagram:
The following...