Modern Python Cookbook

Modern Python Cookbook

Overview of this book

Python is the preferred choice of developers, engineers, data scientists, and hobbyists everywhere. It is a great scripting language that can power your applications and provide great speed, safety, and scalability. By exposing Python as a series of simple recipes, you can gain insight into specific language features in a particular context. Having a tangible context helps make the language or standard library feature easier to understand. This book comes with over 100 recipes on the latest version of Python. The recipes will benefit everyone ranging from beginner to an expert. The book is broken down into 13 chapters that build from simple language concepts to more complex applications of the language. The recipes will touch upon all the necessary Python concepts related to data structures, OOP, functional programming, as well as statistical programming. You will get acquainted with the nuances of Python syntax and how to effectively use the advantages that it offers. You will end the book equipped with the knowledge of testing, web services, and configuration and application integration tips and tricks. The recipes take a problem-solution approach to resolve issues commonly faced by Python programmers across the globe. You will be armed with the knowledge of creating applications with flexible logging, powerful configuration, and command-line options, automated unit tests, and good documentation.

Title Page

Credits

About the Author

About the Reviewers

www.PacktPub.com

Preface

Free Chapter

Numbers, Strings, and Tuples

Introduction

Creating meaningful names and using variables

Working with large and small integers

Choosing between float, decimal, and fraction

Choosing between true division and floor division

Rewriting an immutable string

String parsing with regular expressions

Building complex strings with "template".format()

Building complex strings from lists of characters

Using the Unicode characters that aren't on our keyboards

Encoding strings – creating ASCII and UTF-8 bytes

Decoding bytes – how to get proper characters from some bytes

Using tuples of items

Statements and Syntax

Introduction

Writing Python script and module files – syntax basics

Writing long lines of code

Including descriptions and documentation

Writing better RST markup in docstrings

Designing complex if...elif chains

Designing a while statement which terminates properly

Avoiding a potential problem with break statements

Leveraging the exception matching rules

Avoiding a potential problem with an except: clause

Chaining exceptions with the raise from statement

Managing a context using the with statement

Function Definitions

Introduction

Designing functions with optional parameters

Using super flexible keyword parameters

Forcing keyword-only arguments with the * separator

Writing explicit types on function parameters

Picking an order for parameters based on partial functions

Writing clear documentation strings with RST markup

Designing recursive functions around Python's stack limits

Writing reusable scripts with the script library switch

Built-in Data Structures – list, set, dict

Introduction

Choosing a data structure

Building lists – literals, appending, and comprehensions

Slicing and dicing a list

Deleting from a list – deleting, removing, popping, and filtering

Reversing a copy of a list

Using set methods and operators

Removing items from a set – remove(), pop(), and difference

Creating dictionaries – inserting and updating

Removing from dictionaries – the pop() method and the del statement

Controlling the order of dict keys

Handling dictionaries and sets in doctest examples

Understanding variables, references, and assignment

Making shallow and deep copies of objects

Avoiding mutable default values for function parameters

User Inputs and Outputs

Introduction

Using features of the print() function

Using input() and getpass() for user input

Debugging with "format".format_map(vars())

Using argparse to get command-line input

Using cmd for creating command-line applications

Using the OS environment settings

Basics of Classes and Objects

Introduction

Using a class to encapsulate data and processing

Designing classes with lots of processing

Designing classes with little unique processing

Optimizing small objects with __slots__

Using more sophisticated collections

Extending a collection – a list that does statistics

Using properties for lazy attributes

Using settable properties to update eager attributes

More Advanced Class Design

Introduction

Choosing between inheritance and extension – the is-a question

Separating concerns via multiple inheritance

Leveraging Python's duck typing

Managing global and singleton objects

Using more complex structures – maps of lists

Creating a class that has orderable objects

Defining an ordered collection

Deleting from a list of mappings

Input/Output, Physical Format, and Logical Layout

Introduction

Using pathlib to work with filenames

Reading and writing files with context managers

Replacing a file while preserving the previous version

Reading delimited files with the CSV module

Reading complex formats using regular expressions

Reading JSON documents

Reading XML documents

Reading HTML documents

Upgrading CSV from DictReader to namedtuple reader

Upgrading CSV from a DictReader to a namespace reader

Using multiple contexts for reading and writing files

Testing

Introduction

Using docstrings for testing

Testing functions that raise exceptions

Handling common doctest issues

Creating separate test modules and packages

Combining unittest and doctest tests

Testing things that involve dates or times

Testing things that involve randomness

Mocking external resources

Web Services

Introduction

Implementing web services with WSGI

Using the Flask framework for RESTful APIs

Parsing the query string in a request

Making REST requests with urllib

Parsing the URL path

Parsing a JSON request

Implementing authentication for web services

Application Integration

Introduction

Finding configuration files

Using YAML for configuration files

Using Python for configuration files

Using class-as-namespace for configuration

Designing scripts for composition

Using logging for control and audit output

Combining two applications into one

Combining many applications using the Command design pattern

Managing arguments and configuration in composite applications

Wrapping and combining CLI applications

Wrapping a program and checking the output

Controlling complex sequences of steps

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Reading complex formats using regular expressions

There are many file formats that lack the elegant regularity of a CSV file. One common file format that's rather difficult to parse is a web server log file. These files tend to have complex data without a single separator character or consistent quoting rules.

When we looked at a simplified log file in the Writing generator functions with the yield statement recipe in online Chapter 12, Functional And Reactive Programming Features (link provided in Preface), we saw that the rows look as follows:

[2016-05-08 11:08:18,651] INFO in ch09_r09: Sample Message One 
[2016-05-08 11:08:18,651] DEBUG in ch09_r09: Debugging 
[2016-05-08 11:08:18,652] WARNING in ch09_r09: Something might have gone wrong

There are a variety of punctuation marks used in this file. The csv module can't handle this complexity.

How can we process this kind of data with the elegant simplicity of a CSV file? Can we transform these irregular rows to a more regular data structure...

Modern Python Cookbook

Modern Python Cookbook

Overview of this book

Related Content you might be interested in

Current Title:

Modern Python Cookbook

Reading complex formats using regular expressions