Adding command-line arguments

Often a solution is best structured as a command-line interface that accepts different parameters to change the way it works, for example, scraping a web page from a provided URL or other URL. Python includes a powerful argparse module in the standard library to create rich command-line argument parsing with minimal effort.

Getting ready

The basic use of argparse in a script can be shown in three steps:

1. Define the arguments that your script is going to accept, generating a new parser.
2. Call the defined parser, returning an object with all of the resulting arguments.
3. Use the arguments to call the entry point of your script, which will apply the defined behavior.

Try to use the following general structure for your scripts:

IMPORTS
def main(main parameters):
  DO THINGS
if __name__ == '__main__':
    DEFINE ARGUMENT PARSER
    PARSE ARGS
    VALIDATE OR MANIPULATE ARGS, IF NEEDED
    main(arguments)

The main function makes it easy to know what the entry point for the code is. The section under the if statement is only executed if the file is called directly, but not if it's imported. We'll follow this for all the steps.

How to do it…

1. Create a script that will accept a single integer as a positional argument, and will print a hash symbol that number of times. The recipe_cli_step1.py script is as follows, but note that we are following the structure presented previously, and the main function is just printing the argument:

   import argparse
   def main(number):
       print('#' * number)
   if __name__ == '__main__':
       parser = argparse.ArgumentParser()
       parser.add_argument('number', type=int, help='A number')
       args = parser.parse_args()
   
       main(args.number)

2. Call the script and check how the parameter is presented. Calling the script with no arguments displays the automatic help. Use the automatic argument ‑h to display the extended help:

   $ uv run recipe_cli_step1.py
   usage: recipe_cli_step1.py [-h] number
   recipe_cli_step1.py: error: the following arguments are required: number
   $ uv run recipe_cli_step1.py -h
   usage: recipe_cli_step1.py [-h] number
   
   positional arguments:
     number                A number
   
   options:
     -h, --help            show this help message and exit

3. Calling the script with the extra parameters work as expected

   $ uv run recipe_cli_step1.py 4
   ####
   $ uv run recipe_cli_step1.py not_a_number
   usage: recipe_cli_step1.py [-h] number
   recipe_cli_step1.py: error: argument number: invalid int value: 'not_a_number'

4. Change the script to accept an optional argument for the character to print. The default will be "#". The recipe_cli_step2.py script will look like this:

   import argparse
   def main(character, number):
       print(character * number)
   if __name__ == '__main__':
       parser = argparse.ArgumentParser()
       parser.add_argument('number', type=int, help='A number')
       parser.add_argument('-c', type=str, help='Character to print',
                           default='#')
   args = parser.parse_args()
   main(args.c, args.number)

5. The help is updated, and using the ‑c flag allows us to print different characters:

   $ uv run recipe_cli_step2.py -h
   usage: recipe_cli_step2.py [-h] [-c C] number
   
   positional arguments:
     number                A number
   
   options:
     -h, --help            show this help message and exit
     -c C                  Character to print
   $ uv run recipe_cli_step2.py 4
   ####
   $ uv run recipe_cli_step2.py 5 -c m
   mmmmm

6. Add a flag that changes the behavior when present. The recipe_cli_step3.py script is as follows:

   import argparse
   def main(character, number):
       print(character * number)
   if __name__ == '__main__':
       parser = argparse.ArgumentParser()
       parser.add_argument('number', type=int, help='A number')
       parser.add_argument('-c', type=str, help='Character to print',
                           default='#')
       parser.add_argument('-U', action='store_true', default=False,
                           dest='uppercase',
                           help='Uppercase the character')
       args = parser.parse_args()
       if args.uppercase:
           args.c = args.c.upper()
       main(args.c, args.number)

7. Calling it uppercases the character if the ‑U flag is added:

   $ uv run recipe_cli_step3.py 4
   ####
   $ uv run recipe_cli_step3.py 4 -c f
   ffff
   $ uv run recipe_cli_step3.py 4 -c f -U
   FFFF

How it works…

As described in step 1 of the How to do it… section, the arguments are added to the parser through .add_arguments. Once all of the arguments are defined, calling parse_args() returns an object that contains the results (or exits if there's an error).

Each argument should add a help description, but their behavior can change greatly depending on the details:

• If an argument starts with a ‑, it is considered an optional parameter, like the ‑c argument in step 4. If not, it's a positional argument, like the number argument in step 1.
• For clarity, always define a default value for optional parameters. It will be None if you don't, but this may be confusing. It's a good idea to add default=None to be explicit.
• Remember to always add a help parameter with a description of the parameter; help is automatically generated, as shown in step 2.
• If a type is present, it will be validated, for example, number in step 3. By default, the type will be a string.
• The actions store_true and store_false can be used to generate flags, arguments that don't require any extra parameters. Set the corresponding default value to be the opposite Boolean. This is demonstrated in the U argument in steps 6 and 7.
• The name of the property in the args object will be, by default, the name of the argument (without the dash, if it's present). You can change it with dest. For example, in step 6, the command-line argument ‑U is described as uppercase.

Some arguments can work in coordination with others, as shown in step 3. Prepare the parameters to pass to the main function so they are clear. For example, in step 3, only two parameters are passed, but one may have been modified.

There's more…

You can create long arguments as well with double dashes, for example:

parser.add_argument('-v', '--verbose', action='store_true', default=False,
    help='Enable verbose output')

This will accept both ‑v and ‑‑verbose, and it will store the name verbose.

Adding long names is a good way of making the interface more intuitive and easier to remember.

The main inconvenience when dealing with command-line arguments is that you may end up with too many of them. This creates confusion. Try to make your arguments as independent as possible and don't make too many dependencies between them; otherwise, handling the combinations can be tricky.

In particular, try to not create more than a couple of positional arguments, as they won't have mnemonics. Positional arguments also accept default values, but most of the time, that won't be the expected behavior.

To simplify the usage of the command-line interface, but still allow further customization, you can use environment variables to set default values – as for example in this code (recipe_cli_env.py):

import argparse
import os

default_character = os.environ.get('DEFAULT_CHAR', '#')


def main(character, number):
    print(character * number)


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('number', type=int, help='A number')
    help_msg = f'Character to print (default "{default_character}"). Set with env DEFAULT_CHAR'
    parser.add_argument('-c', type=str, help=help_msg,
                        default=default_character)
    args = parser.parse_args()
    main(args.c, args.number)

Notice how we use the dictionary os.environ to retrieve a value in the environment variable DEFAULT_CHAR. If not present, it will use the value '#'. We store it into default_character, which we use to define the default value in the parser. We include extra information in the help command to make clear how to use the parameters and environment variables.

Because it uses the environment variable, which can be defined in the shell, the need to always specify the character to use when we run it is removed:

$ uv run recipe_cli_env.py 4
####
$ DEFAULT_CHAR=* uv run recipe_cli_env.py 4
****
$ DEFAULT_CHAR=* uv run recipe_cli_env.py -c ^ 5
^^^^^

This pattern is common in command-line utilities and can be used to simplify the call, while allowing for flexibility.

For advanced details on argparse, refer to the Python documentation (https://docs.python.org/3/library/argparse.html).

See also

• The Creating a virtual environment recipe, earlier in this chapter, to learn how to create an environment installing third-party modules.
• The Installing third-party packages recipe, earlier in this chapter, to learn how to install and use external modules in the virtual environment.

Get this book's PDF version and more

Scan the QR code (or go to packtpub.com/unlock). Search for this book by name, confirm the edition, and then follow the steps on the page.

Note: Keep your invoice handy. Purchases made directly from Packt don't require an invoice.