Book Image

Distributed Computing with Python

Book Image

Distributed Computing with Python

Overview of this book

CPU-intensive data processing tasks have become crucial considering the complexity of the various big data applications that are used today. Reducing the CPU utilization per process is very important to improve the overall speed of applications. This book will teach you how to perform parallel execution of computations by distributing them across multiple processors in a single machine, thus improving the overall performance of a big data processing task. We will cover synchronous and asynchronous models, shared memory and file systems, communication between various processes, synchronization, and more.
Table of Contents (15 chapters)
Distributed Computing with Python
Credits
About the Author
About the Reviewer
www.PacktPub.com
Preface
Index

Summary


Through AWS, we saw how cloud providers typically offer compute and storage platforms on a pay-as-you-go model, whereby users (that is, us) only pay for the resources that they actually use.

These platforms can be of significant help in both the development phase as well as in operations for our distributed applications. This is especially true for scalability tests, for instance, and in all situations where ordering and provisioning a set of machines ourselves would take too long or represent too expensive an upfront cost. Not to mention the fact that being able to leverage the vast infrastructure and uptime guarantees of a large cloud provider is no small thing.

At the same time, however, one should be aware of the fact that cloud services are not free. They most definitely are not free economically (apart from maybe the first year of use for AWS). Also, they are not free in terms of time and effort, in that they do not free us from knowing how to administer our resources, install...