Storm makes it easy to deploy and run Python topologies, but developing and testing them in Storm is challenging, whether running in standalone Storm or a full Storm deployment:
Storm launches programs on your behalf—not only your Python code but auxiliary Java processes as well
It controls the Python components' standard input and output channels
The Python programs must respond regularly to heartbeat messages or be shut down by Storm
This makes it difficult to debug Storm topologies using the typical tools and techniques used for other pieces of Python code, such as the common technique of running from the command line and debugging with pdb.
Petrel's mock module helps us with this. It provides a simple, standalone Python container for testing simple topologies and verifying that the expected results are returned.
In Petrel terms, a simple topology is one that only outputs to the default stream and has no branches or loops. The run_simple_topology() assumes that the first component...