Microservices are being featured in every avenue of software development. Microservices are a software development style that has been touted to increase development speed and efficiency while improving software scalability and delivery. This development technique is not unique to any stack and has become extremely popular in Java, .NET, and JavaScript (Node JS) development. While the use of microservices is seen as a pattern, there are several subpatterns that are employed to ensure that the code base is as effective as possible.

This chapter is the first of this 15-chapter book, which will cover design patterns used in microservices. We will be focusing on implementing them using the .NET Core development stack, and you will learn how code can be written and deployed. You will learn about design and coding patterns, third-party tools and environments, and best practices for handling certain scenarios in application development with microservices.

In this chapter, we are going to cover the following topics:

• A deep dive into microservices and its key elements
• Assessing the business need for microservices
• Determining the feasibility of implementing microservices

Traditionally, in software development, applications have developed as a single unit or monolith. All of the components are tightly coupled, and a change to one component threatens to have rippling effects throughout the code base and functionality. This makes long-term maintenance a major concern and can hinder developers from rolling out updates quickly.

Microservices will have you assess that monolith, break it into smaller and more perceivable applications. Each application will relate to a subsection of the larger project, which is a called domain. We will then develop and maintain the code base per application as independent units. Typically, microservices are developed as APIs and may or may not interact with each other to complete operations being carried by users through a unifying user interface. Typically, the microservice architecture comprises a suite of small independent services, which communicate via HTTP (REST APIs) or gRPC (Google Remote Procedure Call). The general notion is that each microservice is autonomous, has a limited scope, and aids in a collectively loosely coupled application.

Building a monolith

Let’s imagine that we need to build a health facility management web application. We need to manage customer information, book appointments, generate invoices, and deliver test results to customers. If we were to itemize all the steps needed to build such an application, key development and scoping activities would include the following:

1. Model the application, and scope the requirements for our customer onboarding, user profiles, and basic documents.
2. Scope the requirements surrounding the process of booking an appointment with a particular doctor. Doctors have schedules and specialties, so we have to present the booking slots accordingly.
3. Create a process flow for when a match is found between a customer and a doctor. Once a match is found, we need to do the following:
   1. Book the doctor’s calendar slot
   2. Generate an invoice
   3. Potentially collect a payment for the visit
   4. Send email notifications to the customer, doctor, and other relevant personnel
4. Model a database (probably relational) to store all this information.
5. Create user interfaces for each screen that both customers and the medical staff will use.

All of this is developed as one application, with one frontend talking to one backend, one database, and one deployment environment. Additionally, we might throw in a few third-party API integrations for payment and email services. This can be load balanced and hosted across multiple servers to mitigate against downtime and increase responsiveness:

Figure 1.1 – Application building

However, this monolithic architecture introduces a few challenges:

• Attempts to extend functionality might have ripple effects through multiple modules and introduce new database and security needs.

Potential solution: Perform thorough unit and integration testing.

• The development team runs the risk of becoming very dependent on a particular stack, making it more difficult to keep the code base modern.

Potential solution: Implement proper versioning strategies and increment them as the technology changes.

• As the code base expands, it becomes more difficult to account for all of the moving parts.

Potential solution: Use clean architectural methods to keep the code base loosely coupled and modular.

The reality is that we can overcome some of these challenges with certain architectural decisions. This all-in-one architecture has been the de facto standard, and frankly, it works. This project architecture is simple, easy enough to scope and develop, and is supported by most, if not all, development stacks and databases. We have been building them for so long that perhaps we have become oblivious to the real challenges that prevail as we try to extend and maintain them in the long term.

Figure 1.2 shows the typical architecture of a monolithic application:

Figure 1.2 – One user interface is served by an API or code library with business logic and is serviced by one database

Now that we have explored the monolithic approach and its potential flaws, let us review a similar application built using microservices.

Building microservices

Now, let us take the same application and conceptualize how it could be architected using microservices. During the design phase, we seek to identify the specific functionalities for each tranche of the application. This is where we identify our domains and subdomains; then, we begin to scope standalone services for each. For example, one domain could be customer management. This service will solely handle the user account and demographic information. Additionally, we could scope bookings and appointments, document management, and finally, payments. This then brings another issue to the foreground: we have dependencies between these three subdomains when we need service independence instead. Using domain-driven design, we then scope out where there are dependencies and identify where we might need to duplicate certain entities. For instance, a customer needs representation in the booking and appointments database as well as payments. This duplication is required if we are using separate databases per service (which is strongly encouraged).

The microservices require us to properly scope the flow of operations that involve multiple services playing a part. For instance, when making a booking, we need to do the following:

6. Retrieve the customer making the booking.
7. Ensure that the preferred time slot is available.
8. If available, generate an invoice.
9. Collect the payment.
10. Confirm the appointment.

That process alone has some back-and-forth processing between the services. Properly orchestrating these service conversations is very critical to having a seamless system and adequately replacing a monolithic approach. Therefore, we introduce various design patterns and approaches to implementing our code and infrastructure. Even though we break potentially complex operations and workflows into smaller and more perceivable chunks, we end up in the same position where the application needs to carry out a specific operation and carry out the original requirements as a whole.

Figure 1.3 shows the typical architecture of a microservices application:

Figure 1.3 – Each microservice is standalone and unifies in a single user interface for user interactions

Now that you are familiar with the differences between the monolithic and microservices approaches, we can explore the pros and cons of using the microservices design pattern.

As we have seen so far, microservices are not easy to author, and they come with many cross-cutting concerns and challenges. It is always important to ask yourself Why? and Do I really need it? before implementing any design patterns.

At a high level, some benefits of this approach are listed as follows:

• Scalability
• Availability
• Development speed
• Improved data storage
• Monitoring
• Deployment

In the following sections, we will dive into the details of each.

Scalability

In the monolithic approach, you scale all or nothing. In microservices, it is easier to scale individual parts of the application and address specific performance gaps as they arise. If a vaccine becomes widely available and customers are encouraged to book an appointment online, then we are sure to experience a large load during the first few weeks. Our customer microservice might not be too affected by that, but we will need to scale our booking and appointments and payments services.

We can scale horizontally, which means that we can allocate more CPU and RAM when the load increases. Alternatively, we can scale vertically by spawning more instances of the service to be load balanced. The better method is relative to the service’s needs. Using the right hosting platforms and infrastructure allows us to automate this process.

Availability

Availability means the probability of a system being operational at a given time. This metric goes hand in hand with the ability to scale, but it also addresses the reliability of the underlying code base and hosting platform. The code base plays a big part in that, so we want to avoid, as much as possible, a single point of failure. A single point of failure affects the entire system if it fails at any point. For example, we will be exploring the gateway pattern, where we will aggregate all services behind one point of entry. For our distributed services to remain available, this gateway must be always online.

This can be achieved by having vertical instances that balance the load and distribute the expected responsiveness of the gateway and, by extension, the underlying services.

Development speed

Given that the application has been broken into domains, developers can focus their efforts on ensuring that their set of features is being developed efficiently. This also contributes to how quickly features can be added, tested, and deployed. It will now become a practical approach to have one team per subdomain. Additionally, it becomes much easier to scope the requirements for a domain and focus on fewer functional requirements for a piece of work. Each team can now be independent and own the service from development to deployment.

This allows Agile and DevOps methodologies to be easier to implement, and it is easier to scope resource requirements per team. Of course, we have seen that services will still need to communicate, so we will still have to orchestrate the integration between the teams. So, while each team is independent, they will still need to make their code and documentation available and easy enough to access. Version control also becomes important since the services will be updated over time, but this must be a managed process.

Improved data storage

Our monolithic application uses one database for the entire application. There are situations where you might end up using one database for multiple microservices, but this is generally discouraged, and a database-per-service approach is preferred. Services must be autonomous and independently developed, deployed, and scaled. This is best accomplished if each service has its own data storage. It makes even more sense when you consider that the type of data being stored might influence the type of data storage that is used. Each service might require a different type of data store, ranging from relational database storage such as Microsoft SQL Server to document-based database storage such as Azure Cosmos DB. We want to ensure that changes to a data store will only affect the associated microservice.

Of course, this will bring its own challenges where data will need to be synchronized across the services. In the monolith, we could wrap all steps inside one transaction, which might lead to performance issues for potentially long-running processes. With microservices, we face the challenge of orchestrating distributed transactions, which also introduces performance risks and threatens the immediate consistency of our data. At this point, we must turn to the concept of eventual consistency. This means that a service publishes an event when its data changes and subscribing services use that event as a signal to update their own data. This approach is made possible through event-sourcing patterns. We accept the risk that, for a period, data might be inconsistent across subdomains. Message queue systems such as Kafka, RabbitMQ, and Azure Service Bus are generally used to accomplish this.

Monitoring

One of the most important aspects of a distributed system is monitoring. This allows us to proactively ensure uptime and mitigate against failures. We need to be able to view the health of our service instances. We also begin to think about how we can centralize logs and performance metrics in a unified manner, sparing us the task of going to each environment manually. Tools such as Kibana, Grafana, or Splunk allow us to create a rich dashboard and visualize all sorts of information about our services.

One very important bit of information is a health check. Sometimes, a microservice instance can be running but is failing to handle requests. For example, it might have run out of database connections. With health checks, we can see a quick snapshot of the service’s health and have that data point returned to the dashboard.

Logging is also a crucial tool for monitoring and troubleshooting. Typically, each microservice would write its own logs to files in its environment. From these logs, we can see information about errors, warnings, information, and debug messages. However, this is not efficient for a distributed system. At this point, we use a log aggregator. This gives us a central area to search and analyze the logs from the dashboards. There are a few log aggregators you can choose from such as LogStash, Splunk, or PaperTrail.

Deployment

Each microservice needs to be independently deployable and scalable. This includes all the security, data storage, and additional assets that our services use. They must all live on physical or virtual servers, whether on-premises or in the cloud. Ideally, each physical server will have its own memory, network, processing, and storage. A virtual infrastructure might have the same physical server with the appropriate resource allocations per service. Here, the idea is that each microservice instance is isolated from the other and will not compete for resources.

Now, each microservice will need its own set of packages and supporting libraries. This then becomes another challenge when provisioning different machines (physical or virtual) and their operating systems. We then seek to simplify this by packaging each microservice as a container image and deploying it as a container. The container will then encapsulate the details of the technology used to build a service and provide all the CPU, memory, and microservice dependencies needed for operation. This makes the microservice easy to move between testing and production environments and provides environment consistency.

Docker is the go-to container management system and works hand in hand with container orchestration services. Orchestration becomes necessary to run multiple containers across multiple machines. We need to start the correct containers at the correct time, handle storage considerations, and address potential container failures. All of these tasks are not practical to handle manually, so we enlist the services of Kubernetes, Docker Swarm, and Marathon to automate these tasks. It is best to have all the deployment steps automated and be as cost-effective as possible.

Then, we look to implement an integrated pipeline that can handle the continuous delivery of our services, with as minimal effort as possible, while maintaining the highest level of consistency possible.

We have explored quite a bit in this section. We reviewed why we might consider using a microservices approach in our development efforts. Also, we investigated some of the most used technologies for this approach. Now, let us turn our attention to justifying our use of microservices.

As we explore the microservices approach, we see where it does address certain things, while introducing a few more concerns. The microservices approach is certainly not a savior for your architectural challenges, and it introduces quite a few complexities. These concerns and complexities are generally addressed using design patterns, and using these patterns can save time and energy.

Throughout this book, we will explore the most common problems we face and look at the design pattern concepts that help us to address these concerns. These patterns can be categorized as follows.

Let us explore what each pattern entails:

• Integration patterns: We have already discussed that microservices will need to communicate. Integration patterns serve to bring consistency to how we accomplish this. Integration patterns govern the technology and techniques that we use to accomplish cross-service communications.
• Database and storage design patterns: We know that we are in for a challenge when it comes to managing data across our distributed services. Giving each service its own database seems easy until we need to ensure that data is kept consistent across the different data stores. There are certain patterns that are pivotal to us maintaining a level of confidence in what we see after each operation.
• Resiliency, security, and infrastructure patterns: These patterns seek to bring calm and comfort to a brewing storm. With all the moving parts that we have identified, it is important to ensure that as many things as possible are automated and consistent in the deployment. Additionally, we want to ensure that security is adequately balanced between the system needs and a good user experience. These patterns help us to ensure that our systems are always performing at peak efficiency.

Next, let us discuss using .NET Core as our development stack for microservices.

Microservices and .NET Core

This book addresses implementing microservices and design patterns using .NET Core. We have mentioned that this architectural style is platform agnostic and has been implemented using several frameworks. Comparably, however, ASP.NET Core makes microservices development very easy and offers many benefits, including cloud integrations, rapid development, and cross-platform support:

• Excellent tooling: The SDK required for .NET development can be installed on any operating system. This is further complemented by their lightweight and open source development tool, Visual Studio Code. You can easily create an API project by running dotnet new webapi on your computer. If you prefer the fully powered Visual Studio IDE, then you might be limited to Windows and macOS. You will have all the tools you need to be successful regardless of the operating system.
• Stability: At the time of writing this book, the latest stable version is .NET 7, with standard term support. The .NET development team is always pushing the envelope and ensuring that reverse compatibility is maintained with each major version release. This makes updating to the next version much less difficult, and you need not worry about too many breaking changes all at once.
• Containerization and scaling: ASP.NET Core applications can easily be mounted on a Docker container, and while this is not necessarily new, we can all appreciate a guaranteed render speed and quality. We can also leverage Kubernetes and easily scale our microservices using all the features of K8s.

.NET development has come a long way, and these are exciting times to push the boundaries of what we can build, using their tools and services.

By now, I hope you have a better idea of what microservices are, why you may or may not end up using this architectural style, and the importance of using design patterns. In each of the chapters of this book, we will explore how to use design patterns to develop a solid and reliable system based on microservices, using .NET Core and various supporting technologies.

We will remain realistic and explore the pros and cons of each of our design decisions and explore how various technologies play integral parts in helping us to tie it all together.

In this chapter, we explored the differences between designing a monolith and microservices, assessed the feasibility of building microservices, and explored why .NET Core is an excellent choice for building microservices

In the next chapter, we will look at implementing the Aggregator Pattern in our microservices application.