Book Image

Python Microservices Development

Book Image

Python Microservices Development

Overview of this book

We often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other. This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you’ll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them. You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker’s role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services. This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microservices.
Table of Contents (20 chapters)
Title Page
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface
Introduction

Microservices pitfalls


As said earlier, building an application with microservices has a lot of benefits, but it's not a silver bullet by all means.

You need to be aware of these main problems you might have to deal with when coding microservices:

  • Illogical splitting
  • More network interactions
  • Data storing and sharing
  • Compatibility issues
  • Testing

These issues will be covered in detail in the following sections.

Illogical splitting

The first issue of a microservice architecture is how it gets designed. There's no way a team can come up with the perfect microservice architecture in the first shot. Some microservices like the PDF generator are an obvious use case. But as soon as you deal with the business logic, there are good chances that your code will move around before you get a good grasp of how to split things into the right set of microservices.

The design needs to mature with some try-and-fail cycles. And adding and removing microservices can be more painful than refactoring a monolithic application.

You can mitigate this problem by avoiding splitting your app in microservices if the split is not evident.

Premature splitting is the root of all evil.

If there's any doubt that the split makes sense, keeping the code in the same app is the safe bet. It's always easier to split apart some of the code into a new microservice later than to merge back to two microservices in the same code base because the decision turned out to be wrong.

For instance, if you always have to deploy two microservices together, or if one change in a microservice impacts the data model of another one, the odds are that you did not split the application correctly, and that those two services should be reunited.

More network interactions

The second problem is the amount of network interactions added to build the same application. In the monolithic version, even if the code gets messy, everything happens in the same process, and you can send back the result without having to call too many backend services to build the actual response.

That requires extra attention on how each backend service is called, and raises a lot of questions like the following:

  • What happens when the Booking UI cannot reach the PDF reporting service because of a network split or a laggy service?
  • Does the Booking UI call the other services synchronously or asynchronously?
  • How will that impact the response time?

We will need to have a solid strategy to be able to answer all those questions, and we will address those in Chapter 5, Interacting with Other Services.

Data storing and sharing

Another problem is data storing and sharing. An effective microservice needs to be independent of other microservices, and ideally, should not share a database. What does this mean for our hotel booking app?

Again, that raises a lot of questions such as the following:

  • Do we use the same users' IDs across all databases, or do we have independent IDs in each service and keep it as a hidden implementation detail?
  • Once a user is added to the system, do we replicate some of her information in other services databases via strategies like data pumping, or is that overkill?
  • How do we deal with data removal?

These are hard questions to answer, and there are many different ways to solve those problems, as we'll learn throughout the book.

Note

Avoiding data duplication as much as possible while keeping microservices in isolation is one of the biggest challenges in designing microservices-based applications.

Compatibility issues

Another problem happens when a feature change impacts several microservices. If a change affects in a backward incompatible way the data that travels between services, you're in for some trouble.

Can you deploy your new service, and will it work with older versions of other services? Or do you need to change and deploy several services at once? Does it mean you've just stumbled on some services that should probably be merged back together?

A good versioning and API design hygiene help to mitigate those issues, as we will discover in the second part of the book when we'll build our application.

Testing

Last, when you want to do some end-to-end tests and deploy your whole app, you now have to deal with many bricks. You need to have a robust and agile deployment process to be efficient. You need to be able to play with your whole application when you develop it. You can't fully test things out with just one piece of the puzzle.

Hopefully, there are now many tools to facilitate deployments of applications that are built with several components, as we will learn about throughout this book. And all those tools probably helped in the success and adoption of microservices and vice versa.

Note

Microservices-style architecture boosts deployment tools innovation, and deployment tools lower the bar for the approval of microservices-style architecture.

The pitfalls of using microservices can be summarized as follows:

  • Premature splitting of an application into microservices can lead to architectural problems
  • Network interactions between microservices add weaknesses spots and additional overhead
  • Testing and deploying microservices can be complex
  • And the biggest challenge--data sharing between microservices is hard

You should not worry too much about all the pitfalls described in this section for now.

They may seem overwhelming, and the traditional monolithic application may look like a safer bet, but in the long term, splitting your project into microservices will make many of your tasks, as a developer or as an Operation person (Ops), easier.