Book Image

Distributed Computing with Python

Book Image

Distributed Computing with Python

Overview of this book

CPU-intensive data processing tasks have become crucial considering the complexity of the various big data applications that are used today. Reducing the CPU utilization per process is very important to improve the overall speed of applications. This book will teach you how to perform parallel execution of computations by distributing them across multiple processors in a single machine, thus improving the overall performance of a big data processing task. We will cover synchronous and asynchronous models, shared memory and file systems, communication between various processes, synchronization, and more.
Table of Contents (15 chapters)
Distributed Computing with Python
Credits
About the Author
About the Reviewer
www.PacktPub.com
Preface
Index

Chapter 8. The Road Ahead

This has been a quick, and hopefully interesting journey into the world of parallel and distributed computing with Python. What this book has really tried to do is convince you that writing a small- to medium-sized distributed applications with Python is not only within the reach of most developers, but it is also quite simple.

Sure, there are a lot of moving parts in even a simple distributed application—possibly, many more than in a regular monolithic application. There are, more critically, also a lot more failure scenarios and a lot many things happening at the same time on different machines.

Luckily, however, there are several high-quality, well-tested Python libraries and frameworks that you can easily leverage to write distributed systems in a much easier way than most of us might think.

Furthermore, parallel and distributed computing is rapidly becoming mainstream, and with the introduction of many-core CPUs (that is, CPUs with a relatively high number of...