Book Image

Parallel Programming with Python

By : Jan Palach, Jan Palach V Cruz da Silva
Book Image

Parallel Programming with Python

By: Jan Palach, Jan Palach V Cruz da Silva

Overview of this book

Table of Contents (16 chapters)
Parallel Programming with Python
About the Author
About the Reviewers

Understanding Celery

Celery is a framework that offers mechanisms to lessen difficulties while creating distributed systems. The Celery framework works with the concept of distribution of work units (tasks) by exchanging messages among the machines that are interconnected as a network, or local workers. A task is the key concept in Celery; any sort of job we must distribute has to be encapsulated in a task beforehand.

Why use Celery?

We could justify the use of Celery by listing some of its positive points:

  • It distributes tasks in a transparent way among workers that are spread over the Internet, or local workers

  • It changes, in a simple way, the concurrence of workers through setup (processes, threads, Gevent, Eventlet)

  • It supports synchronous, asynchronous, periodic, and scheduled tasks

  • It re-executes tasks in case of errors


It is common for some developers to claim that synchronous tasks are the same as real-time tasks. This is an unnecessary confusion as the concepts are totally different...