Book Image

Python 3 Text Processing with NLTK 3 Cookbook

By : Jacob Perkins
Book Image

Python 3 Text Processing with NLTK 3 Cookbook

By: Jacob Perkins

Overview of this book

Table of Contents (17 chapters)
Python 3 Text Processing with NLTK 3 Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Penn Treebank Part-of-speech Tags
Index

Parallel list processing with execnet


This recipe presents a pattern for using execnet to process a list in parallel. It's a function pattern for mapping each element in the list to a new value, using execnet to do the mapping in parallel.

How to do it...

First, we need to decide exactly what we want to do. In this example, we'll just double integers, but we could do any pure computation. Following is the remote_double.py module, which will be executed by execnet. It receives a 2-tuple of (i, arg), assumes arg is a number, and sends back (i, arg*2). The need for i will be explained in the next section.

if __name__ == '__channelexec__':
  for (i, arg) in channel:
    channel.send((i, arg * 2))

To use this module to double every element in a list, we import the plists module (explained in the How it works... section) and call plists.map() with the remote_double module, and a list of integers to double.

>>> import plists, remote_double
>>> plists.map(remote_double, range(10))
...