Locating ObjectGrid containers in different cities is a common use. Consider our ObjectGrid deployment policy file with zones named rack1, rack2, rack3, and rack4 running in New York City (NYC), and zones named rack1a and rack2a running in Boston. Working with the grid a client in NYC will read from primary and synchronous replicas with very low latency. A client will only read from an asynchronous replica in Boston if no shard in NYC is available to fulfill the request.
However, a grid client in Boston, will also prefer to read from primary and synchronous replicas in NYC. Each read and write request will make an RPC, not to the asynchronous replicas in Boston, but to the shards in NYC. Making this RPC for every grid request kills throughput because of the latency involved in going to a shard located in NYC. The Boston-based asynchronous replicas are essentially backups, in case all shards in NYC become unavailable.
We stress that locality of reference is very important in...