Book Image

Microservices Deployment Cookbook

By : Vikram Murugesan
Book Image

Microservices Deployment Cookbook

By: Vikram Murugesan

Overview of this book

This book will help any team or organization understand, deploy, and manage microservices at scale. It is driven by a sample application, helping you gradually build a complete microservice-based ecosystem. Rather than just focusing on writing a microservice, this book addresses various other microservice-related solutions: deployments, clustering, load balancing, logging, streaming, and monitoring. The initial chapters offer insights into how web and enterprise apps can be migrated to scalable microservices. Moving on, you’ll see how to Dockerize your application so that it is ready to be shipped and deployed. We will look at how to deploy microservices on Mesos and Marathon and will also deploy microservices on Kubernetes. Next, you will implement service discovery and load balancing for your microservices. We’ll also show you how to build asynchronous streaming systems using Kafka Streams and Apache Spark. Finally, we wind up by aggregating your logs in Kafka, creating your own metrics, and monitoring the metrics for the microservice.
Table of Contents (15 chapters)
Microservices Deployment Cookbook
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Improving the performance of the Spark job


In the previous recipe, we wrote a simple Spark job that filters out invalid geolocations and pushes the valid geolocations into a Kafka topic. In this recipe, we will see how we can improve the performance of our Spark job.

How to do it...

There are several ways in which you can improve the performance of your Spark job. There are a lot many configurations that Spark provides that can be tweaked to achieve desired performance. For example, based on the amount of data that your topic receives, you could change the batch duration of your stream. Also, deploying your Spark job on a Mesos or YARN cluster opens up a lot of opportunities for performance improvement. In fact, running your Spark job in local standalone mode will not help you assess the performance of your Spark job. The real test for a Spark job is when it is executed on a cluster. Each Spark job requires a certain amount of resources for execution, be it CPU or memory.

Earlier in the book...