A microservice architecture introduces well-defined boundaries, which makes it possible to isolate failures within the boundaries. But being like other distributed systems, there is likely a chance of failure at the application level. To minimize the impact, we need to design fault-tolerant microservices, which react in a predefined way to certain types of failure. While adapting to microservice architecture, we add one more network layer to communicate with rather than in-memory method calls, which introduces extra latency and one more layer to manage. Given next are a few considerations that if handled with care while designing microservices for failure, will benefit the system in the long run.
Microservices architecture allows you to isolate failures, thus enabling you to isolate the failures and get graceful degradation as failures are contained within the boundaries of the service and are not cascaded. For example, in social networking sites, the messaging service may go down, but that won't stop the end users from using social networks. They can still browse posts, share statuses, check-in locations, and so on. Services should be made such that they adhere to certain SLAs. If a microservice stops meeting its SLA, then that service should be restored to back up. Netflix's Hystrix is based on the same principle.
Introducing change without any governance can be a huge problem. In a distributed system, services depend on each other. So when you introduce a new change, the utmost consideration should be given as if any side or unwanted effects are introduced, then its effect should be minimal. Various change management strategies and automatic rollout options should be available. Also, proper governance should be there in code management. Development should be done via TDD or BDD, and if the agreed percentage is met upon, only then should it be rolled out. Releases should be done gradually. One useful strategy is the blue-green or red-black deployment strategy, wherein you run two production environments. You rollout the change in only one environment and point out the load balancer to a newer version only after your change is verified. This is more likely when maintaining a staging environment.
Depending on business requirements, the microservice instance can start, restart, stop on some failure, run low on memory, and auto-scale, which may make it temporarily or permanently unavailable. Therefore, the architecture and framework should be designed accordingly. For example, a Node.js server, being single-threaded, stops immediately in the case of failure, but using graceful tools such as PM2 forever keeps them running. A gateway should be introduced that will be the only point of contact for the microservice consumer. The gateway can be a load balancer that should skip unhealthy microservices instances. The load balancer should be able to collect health information metrics and route traffic accordingly, it should smartly analyze the traffic on any particular microservice, and it should trigger auto-scaling if needed.
Self-curing design can help the system to recover from disasters. Microservices implementation should be able to recover lost services and functionality automatically. Tools such as Docker restart services whenever they fail. Netflix provides wide tools as an orchestration layer to achieve self-healing. Eureka service registry and Hystrix circuit breaker are commonly used. Circuit breakers make your service calls more resilient. They track each microservice endpoint's status. Whenever timeout is encountered, Hystrix breaks the connection, triggers the need for curing that microservice, and reverts to some fail-safe strategy. Kubernates is another option. If a pod or any container inside the pod goes down, Kubernates brings up the system and maintains the replica set intact.
Failover caching helps to provide necessary data whenever there are temporary failures or some glitches. The cache layer should be designed so that it can smartly decide how long the cache can be used in a normal situation or during failover situations. Setting cache standard response headers in HTTP can be used. The max-age header specifies the amount of time a resource will be considered fresh. The stale-if-error header determines how long the resource should be served from the cache. You can also use libraries such as Memcache, Redis, and so on.
Due to its self-healing capabilities, a microservice usually gets up and running in no time. Microservice architecture should have retry logic until condition capabilities, as we can expect that the service will recover or the load balancer will redirect the service request to another healthy instance. Frequent retries can also have a huge impact on the system. A general idea is increasing the waiting time between retries after each failure. Microservices should be able to handle idempotency issues; let's say you are retrying to purchase an order, then there shouldn't be double purchases on the customer. Now, let's take time to revisit the microservice concept and understand the most common questions asked about microservice architecture.