Book Image

Python Programming Blueprints

By : Daniel Furtado, Marcus Pennington
Book Image

Python Programming Blueprints

By: Daniel Furtado, Marcus Pennington

Overview of this book

Python is a very powerful, high-level, object-oriented programming language. It's known for its simplicity and huge community support. Python Programming Blueprints will help you build useful, real-world applications using Python. In this book, we will cover some of the most common tasks that Python developers face on a daily basis, including performance optimization and making web applications more secure. We will familiarize ourselves with the associated software stack and master asynchronous features in Python. We will build a weather application using command-line parsing. We will then move on to create a Spotify remote control where we'll use OAuth and the Spotify Web API. The next project will cover reactive extensions by teaching you how to cast votes on Twitter the Python way. We will also focus on web development by using the famous Django framework to create an online game store. We will then create a web-based messenger using the new Nameko microservice framework. We will cover topics like authenticating users and, storing messages in Redis. By the end of the book, you will have gained hands-on experience in coding with Python.
Table of Contents (17 chapters)
Title Page
Copyright and Credits
Dedication
Contributors
Packt Upsell
Preface
Index

Getting the user's input with ArgumentParser


Before we run our application for the first time, we need to add the application's entry point. The entry point is the first code that will be run when our application is executed. 

We want to give the users of our application the best user experience possible, so the first features that we need to add are the ability to receive and parse command line arguments, perform argument validation, set arguments when needed, and, last but not least, show an organized and informative help system so the users can see which arguments can be used and how to use the application. 

Sounds like tedious work, right?

Luckily, Python has batteries included and the standard library contains a great module that allows us to implement this in a very simple way; the module is called argparse.

Another feature that would be good to have is for our application to be easy to distribute to our users. One approach is to create a __main__.py file in the weatherterm module directory, and you can run the module as a regular script. Python will automatically run the __main__.py file, like so:

$ python -m weatherterm

Another option is to zip the entire application's directory and execute the Python passing the name of the ZIP file instead. This is an easy, fast, and simple way to distribute our Python programs.

There are many other ways of distributing your programs, but they are beyond the scope of this book; I just wanted to give you some examples of the usage of the __main__.py file.

With that said, let's create a __main__.py file inside of the weatherterm directory with the following content:

import sys
from argparse import ArgumentParser

from weatherterm.core import parser_loader
from weatherterm.core import ForecastType
from weatherterm.core import Unit


def _validate_forecast_args(args):
    if args.forecast_option is None:
        err_msg = ('One of these arguments must be used: '
                   '-td/--today, -5d/--fivedays, -10d/--tendays, -
                    w/--weekend')
        print(f'{argparser.prog}: error: {err_msg}', 
        file=sys.stderr)
        sys.exit()

parsers = parser_loader.load('./weatherterm/parsers')

argparser = ArgumentParser(
    prog='weatherterm',
    description='Weather info from weather.com on your terminal')

required = argparser.add_argument_group('required arguments')

required.add_argument('-p', '--parser',
                      choices=parsers.keys(),
                      required=True,
                      dest='parser',
                      help=('Specify which parser is going to be  
                       used to '
                            'scrape weather information.'))

unit_values = [name.title() for name, value in Unit.__members__.items()]

argparser.add_argument('-u', '--unit',
                       choices=unit_values,
                       required=False,
                       dest='unit',
                       help=('Specify the unit that will be used to 
                       display '
                             'the temperatures.'))

required.add_argument('-a', '--areacode',
                      required=True,
                      dest='area_code',
                      help=('The code area to get the weather 
                       broadcast from. '
                            'It can be obtained at 
                              https://weather.com'))

argparser.add_argument('-v', '--version',
                       action='version',
                       version='%(prog)s 1.0')

argparser.add_argument('-td', '--today',
                       dest='forecast_option',
                       action='store_const',
                       const=ForecastType.TODAY,
                       help='Show the weather forecast for the 
                       current day')

args = argparser.parse_args()

_validate_forecast_args(args)

cls = parsers[args.parser]

parser = cls()
results = parser.run(args)

for result in results:
    print(results)

The weather forecast options (today, five days, ten days, and weekend forecast) that our application will accept will not be required; however, at least one option must be provided in the command line, so we create a simple function called _validate_forecast_args to perform this validation for us. This function will show a help message and exit the application.

First, we get all the parsers available in the weatherterm/parsers directory. The list of parsers will be used as valid values for the parser argument.

It is the ArgumentParser object that does the job of defining the parameters, parsing the values, and showing help, so we create an instance of ArgumentParser and also create an argument group for the required parameters. This will make the help output look much nicer and organized.

In order to make the parameters and the help output more organized, we are going to create a group within the ArgumentParser object. This group will contain all the required arguments that our application needs. This way, the users of our application can easily see which parameters are required and the ones that are not required.

We achieve this with the following statement:

required = argparser.add_argument_group('required arguments')

After creating the argument group for the required arguments, we get a list of all members of the enumeration Unit and use the title() function to make only the first letter a capital letter.

Now, we can start adding the arguments that our application will be able to receive on the command line. Most argument definitions use the same set of keyword arguments, so I will not be covering all of them.

The first argument that we will create is --parser or -p:

required.add_argument('-p', '--parser',
                      choices=parsers.keys(),
                      required=True,
                      dest='parser',
                      help=('Specify which parser is going to be 
                       used to '
                            'scrape weather information.'))

Let's break down every parameter of the add_argument  used when creating the parser flag:

  • The first two parameters are the flags. In this case, the user passes a value to this argument using either -p or --parser in the command line, for example, --parser WeatherComParser.
  • The choices parameter specifies a list of valid values for that argument that we are creating. Here, we are using parsers.keys(), which will return a list of parser names. The advantage of this implementation is that if we add a new parser, it will be automatically added to this list, and no changes will be required in this file.
  • The required parameter, as the name says, specifies if the argument will be required or not.
  • The dest parameter specifies the name of the attribute to be added to the resulting object of the parser argument. The object returned by parser_args() will contain an attribute called parser with the value that we passed to this argument in the command line.
  • Finally, the help parameter is the argument's help text, shown when using the -h or --help flag.

Moving on to the --today argument:

argparser.add_argument('-td', '--today',
                       dest='forecast_option',
                       action='store_const',
                       const=ForecastType.TODAY,
                       help='Show the weather forecast for the 
                       current day')

Heren we have two keyword arguments that we haven't seen before, action and const.

Actions can be bound to the arguments that we create and they can perform many things. The argparse module contains a great set of actions, but if you need to do something specific, you can create your own action that will meet your needs. Most actions defined in the argparse module are actions to store values in the parse result's object attributes.

In the previous code snippet, we use the store_const action, which will store a constant value to an attribute in the object returned by parse_args().

We also used the keyword argument const, which specifies the constant default value when the flag is used in the command line.

Remember that I mentioned that it is possible to create custom actions? The argument unit is a great use case for a custom action. The choices argument is just a list of strings, so we use this comprehension to get the list of names of every item in the Unit enumeration, as follows:

unit_values = [name.title() for name, value in Unit.__members__.items()]

required.add_argument('-u', '--unit',
                      choices=unit_values,
                      required=False,
                      dest='unit',
                      help=('Specify the unit that will be used to 
                       display '
                            'the temperatures.'))

The object returned by parse_args() will contain an attribute called unit with a string value (Celsius or Fahrenheit), but this is not exactly what we want. Wouldn't it be nice to have the value as an enumeration item instead? We can change this behavior by creating a custom action.

First, add a new file named set_unit_action.py in the weatherterm/core directory with the following contents:

from argparse import Action

from weatherterm.core import Unit


class SetUnitAction(Action):

    def __call__(self, parser, namespace, values, option_string=None):
        unit = Unit[values.upper()]
        setattr(namespace, self.dest, unit)

This action class is very simple; it just inherits from argparse.Action and overrides the __call__ method, which will be called when the argument value is parsed. This is going to be set to the destination attribute.

The parser parameter will be an instance of ArgumentParser. The namespace is an instance of argparser.Namespace and it is just a simple class containing all the attributes defined in the ArgumentParser object. If you inspect this parameter with the debugger, you will see something similar to this:

Namespace(area_code=None, fields=None, forecast_option=None, parser=None, unit=None)

The values parameter is the value that the user has passed on the command line; in our case, it can be either Celsius or Fahrenheit. Lastly, the option_string parameter is the flag defined for the argument. For the unit argument, the value of option_string will be -u.

Fortunately, enumerations in Python allow us to access their members and attributes using item access:

Unit[values.upper()]

Verifying this in Python REPL, we have:

>>> from weatherterm.core import Unit
>>> Unit['CELSIUS']
<Unit.CELSIUS: 'CELSIUS'>
>>> Unit['FAHRENHEIT']
<Unit.FAHRENHEIT: 'FAHRENHEIT'>

After getting the correct enumeration member, we set the value of the property specified by self.dest in the namespace object. That is much cleaner and we don't need to deal with magic strings.

With the custom action in place, we need to add the import statement in the __init__.py file in the weatherterm/core directory:

from .set_unit_action import SetUnitAction

Just include the line above at the end of the file. Then, we need to import it into the __main__.py file,  like so:

from weatherterm.core import SetUnitAction

And we are going to add the action keyword argument in the definition of the unit argument and set it to SetUnitAction, like so:

required.add_argument('-u', '--unit',
                      choices=unit_values,
                      required=False,
                      action=SetUnitAction,
                      dest='unit',
                      help=('Specify the unit that will be used to 
                       display '
                            'the temperatures.'))

So, when the user of our application uses the flag -u for Celsius, the value of the attribute unit in the object returned by the parse_args()  function will be:

<Unit.CELSIUS: 'CELSIUS'>

The rest of the code is very straightforward; we invoke the parse_args function to parse the arguments and set the result in the args variable. Then, we use the value of args.parser (the name of the selected parser) and access that item in the parser's dictionary. Remember that the value is the class type, so we create an instance of the parser, and lastly, invoke the method run, which will kick off website scraping.