Book Image

Python Automation Cookbook

By : Jaime Buelta
Book Image

Python Automation Cookbook

By: Jaime Buelta

Overview of this book

Have you been doing the same old monotonous office work over and over again? Or have you been trying to find an easy way to make your life better by automating some of your repetitive tasks? Through a tried and tested approach, understand how to automate all the boring stuff using Python. The Python Automation Cookbook helps you develop a clear understanding of how to automate your business processes using Python, including detecting opportunities by scraping the web, analyzing information to generate automatic spreadsheets reports with graphs, and communicating with automatically generated emails. You’ll learn how to get notifications via text messages and run tasks while your mind is focused on other important activities, followed by understanding how to scan documents such as résumés. Once you’ve gotten familiar with the fundamentals, you’ll be introduced to the world of graphs, along with studying how to produce organized charts using Matplotlib. In addition to this, you’ll gain in-depth knowledge of how to generate rich graphics showing relevant information. By the end of this book, you’ll have refined your skills by attaining a sound understanding of how to identify and correct problems to produce superior and reliable systems.
Table of Contents (12 chapters)

Using a third-party tool—parse

While manually parsing data, as seen in the previous recipe, works very well for small strings, it can be very laborious to tweak the exact formula to work with a variety of input. What if the input has an extra dash sometimes? Or it has a variable length header depending on the size of one of the fields?

A more advanced option is to use regular expressions, as we'll see in the next recipe. But there's a great module in Python called parse ( that allows us to reverse format strings. It is a fantastic tool, that's powerful, easy to use, and greatly improves the readability of the code.

Getting ready

Add the parse module to the requirements.txt file in our virtual environment and reinstall the dependencies, as shown in the Creating a virtual environment recipe.

The requirements.txt file should look like this:


Then, reinstall the modules in the virtual environment:

$ pip install -r requirements.txt
Collecting parse==1.8.2 (from -r requirements.txt (line 3))
Using cached
Installing collected packages: parse
Running install for parse ... done
Successfully installed parse-1.8.2

How to do it...

  1. Import the parse function:
>>> from parse import parse
  1. Define the log to parse, in the same format as in the Extracting data from structured strings recipe:
>>> LOG = '[2018-05-06T12:58:00.714611] - SALE - PRODUCT: 1345 - PRICE: $09.99'
  1. Analyze it and describe it as you'll do when trying to print it, like this:
>>> FORMAT = '[{date}] - SALE - PRODUCT: {product} - PRICE: ${price}'
  1. Run parse and check the results:
>>> result = parse(FORMAT, LOG)
>>> result
<Result () {'date': '2018-05-06T12:58:00.714611', 'product': '1345', 'price': '09.99'}>
>>> result['date']
>>> result['product']
>>> result['price']
  1. Note the results are all strings. Define the types to be parsed:
>>> FORMAT = '[{date:ti}] - SALE - PRODUCT: {product:d} - PRICE: ${price:05.2f}'
  1. Parse once again:
>>> result = parse(FORMAT, LOG)
>>> result
<Result () {'date': datetime.datetime(2018, 5, 6, 12, 58, 0, 714611), 'product': 1345, 'price': 9.99}>
>>> result['date']
datetime.datetime(2018, 5, 6, 12, 58, 0, 714611)
>>> result['product']
>>> result['price']
  1. Define a custom type for the price to avoid issues with the float type:
>>> from decimal import Decimal
>>> def price(string):
... return Decimal(string)
>>> FORMAT = '[{date:ti}] - SALE - PRODUCT: {product:d} - PRICE: ${price:price}'
>>> parse(FORMAT, LOG, {'price': price})
<Result () {'date': datetime.datetime(2018, 5, 6, 12, 58, 0, 714611), 'product': 1345, 'price': Decimal('9.99')}>

How it works...

The parse module allows us to define a format, such as string, that reverses the format method when parsing values. A lot of the concepts that we discussed when creating strings applies here—put values in brackets, define the type after a colon, and so on.

By default, as seen in step 4, the values are parsed as strings. This is a good starting point when analyzing text. The values can be parsed into more useful native types, as shown in steps 5 and 6 in the How to do it... section. Please note that while most of the parsing types are the same as the ones in the Python Format Specification mini-language, there are some others available, such as ti for timestamps in ISO format.

If native types are not enough, our own parsing can be defined, as demonstrated in step 7 in the How to do it... section. Note that the definition of the price function gets a string and returns the proper format, in this case a Decimal type.

All the issues about floats and price information described in the There's more section of the Extracting data from structured strings recipe apply here as well.

There's more...

The timestamp can also be translated into a delorean object for consistency. Also, delorean objects carry over timezone information. Adding the same structure as in the previous recipe gives the following object, which is capable of parsing logs:

class PriceLog(object):
def __init__(self, timestamp, product_id, price):
self.timestamp = timestamp
self.product_id = product_id
self.price = price
def __repr__(self):
return '<PriceLog ({}, {}, {})>'.format(self.timestamp,
def parse(cls, text_log):
Parse from a text log with the format
[<Timestamp>] - SALE - PRODUCT: <product id> - PRICE: $<price>
to a PriceLog object
def price(string):
return Decimal(string)
def isodate(string):
return delorean.parse(string)
FORMAT = ('[{timestamp:isodate}] - SALE - PRODUCT: {product:d} - '
'PRICE: ${price:price}')
formats = {'price': price, 'isodate': isodate}
result = parse.parse(FORMAT, text_log, formats)
return cls(timestamp=result['timestamp'],

So, parsing it returns similar results:

>>> log = '[2018-05-06T14:58:59.051545] - SALE - PRODUCT: 827 - PRICE: $22.25'
>>> PriceLog.parse(log)
<PriceLog (Delorean(datetime=datetime.datetime(2018, 6, 5, 14, 58, 59, 51545), timezone='UTC'), 827, 22.25)>

This code is contained in the GitHub file Chapter01/

All parse supported types can be found in the documentation at

See also

  • The Extracting data from structured strings recipe
  • The Introducing regular expressions recipe
  • The Going deeper into regular expressions recipe