Modern Python Cookbook - Second Edition

By : Steven F. Lott

Modern Python Cookbook - Second Edition

By: Steven F. Lott

Overview of this book

Python is the preferred choice of developers, engineers, data scientists, and hobbyists everywhere. It is a great language that can power your applications and provide great speed, safety, and scalability. It can be used for simple scripting or sophisticated web applications. By exposing Python as a series of simple recipes, this book gives you insight into specific language features in a particular context. Having a tangible context helps make the language or a given standard library feature easier to understand. This book comes with 133 recipes on the latest version of Python 3.8. The recipes will benefit everyone, from beginners just starting out with Python to experts. You'll not only learn Python programming concepts but also how to build complex applications. The recipes will touch upon all necessary Python concepts related to data structures, object oriented programming, functional programming, and statistical programming. You will get acquainted with the nuances of Python syntax and how to effectively take advantage of it. By the end of this Python book, you will be equipped with knowledge of testing, web services, configuration, and application integration tips and tricks. You will be armed with the knowledge of how to create applications with flexible logging, powerful configuration, command-line options, automated unit tests, and good documentation.

Preface

What you need for this book

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Numbers, Strings, and Tuples

Working with large and small integers

Choosing between float, decimal, and fraction

Choosing between true division and floor division

Rewriting an immutable string

String parsing with regular expressions

Building complex strings with f-strings

Building complicated strings from lists of characters

Using the Unicode characters that aren't on our keyboards

Encoding strings – creating ASCII and UTF-8 bytes

Decoding bytes – how to get proper characters from some bytes

Using tuples of items

Using NamedTuples to simplify item access in tuples

Free Chapter

Statements and Syntax

Writing Python script and module files – syntax basics

Writing long lines of code

Including descriptions and documentation

Writing better RST markup in docstrings

Designing complex if...elif chains

Saving intermediate results with the := "walrus"

Avoiding a potential problem with break statements

Leveraging exception matching rules

Avoiding a potential problem with an except: clause

Concealing an exception root cause

Managing a context using the with statement

Function Definitions

Function parameters and type hints

Designing functions with optional parameters

Designing type hints for optional parameters

Using super flexible keyword parameters

Forcing keyword-only arguments with the * separator

Defining position-only parameters with the / separator

Writing hints for more complex types

Picking an order for parameters based on partial functions

Writing clear documentation strings with RST markup

Designing recursive functions around Python's stack limits

Writing testable scripts with the script-library switch

Built-In Data Structures Part 1: Lists and Sets

Choosing a data structure

Building lists – literals, appending, and comprehensions

Slicing and dicing a list

Deleting from a list – deleting, removing, popping, and filtering

Writing list-related type hints

Reversing a copy of a list

Building sets – literals, adding, comprehensions, and operators

Removing items from a set – remove(), pop(), and difference

Writing set-related type hints

Built-In Data Structures Part 2: Dictionaries

Creating dictionaries – inserting and updating

Removing from dictionaries – the pop() method and the del statement

Controlling the order of dictionary keys

Writing dictionary-related type hints

Understanding variables, references, and assignment

Making shallow and deep copies of objects

Avoiding mutable default values for function parameters

User Inputs and Outputs

Using the features of the print() function

Using input() and getpass() for user input

Debugging with f"{value=}" strings

Using argparse to get command-line input

Using cmd to create command-line applications

Using the OS environment settings

Basics of Classes and Objects

Using a class to encapsulate data and processing

Essential type hints for class definitions

Designing classes with lots of processing

Using typing.NamedTuple for immutable objects

Using dataclasses for mutable objects

Using frozen dataclasses for immutable objects

Optimizing small objects with __slots__

Using more sophisticated collections

Extending a built-in collection – a list that does statistics

Using properties for lazy attributes

Creating contexts and context managers

Managing multiple contexts with multiple resources

More Advanced Class Design

Choosing between inheritance and composition – the "is-a" question

Separating concerns via multiple inheritance

Leveraging Python's duck typing

Managing global and singleton objects

Using more complex structures – maps of lists

Creating a class that has orderable objects

Improving performance with an ordered collection

Deleting from a list of complicated objects

Functional Programming Features

Introduction

Writing generator functions with the yield statement

Applying transformations to a collection

Using stacked generator expressions

Picking a subset – three ways to filter

Summarizing a collection – how to reduce

Combining the map and reduce transformations

Implementing "there exists" processing

Creating a partial function

Simplifying complex algorithms with immutable data structures

Writing recursive generator functions with the yield from statement

Input/Output, Physical Format, and Logical Layout

Using pathlib to work with filenames

Replacing a file while preserving the previous version

Reading delimited files with the CSV module

Using dataclasses to simplify working with CSV files

Reading complex formats using regular expressions

Reading JSON and YAML documents

Reading XML documents

Reading HTML documents

Refactoring a .csv DictReader as a dataclass reader

Testing

Test tool setup

Using docstrings for testing

Testing functions that raise exceptions

Handling common doctest issues

Unit testing with the unittest module

Combining unittest and doctest tests

Unit testing with the pytest module

Combining pytest and doctest tests

Testing things that involve dates or times

Testing things that involve randomness

Mocking external resources

Web Services

Defining the card model

Using the Flask framework for RESTful APIs

Parsing the query string in a request

Making REST requests with urllib

Parsing the URL path

Parsing a JSON request

Implementing authentication for web services

Application Integration: Configuration

Finding configuration files

Using YAML for configuration files

Using Python for configuration files

Using class-as-namespace for configuration

Designing scripts for composition

Using logging for control and audit output

Application Integration: Combination

Combining two applications into one

Combining many applications using the Command Design Pattern

Managing arguments and configuration in composite applications

Wrapping and combining CLI applications

Wrapping a program and checking the output

Controlling complex sequences of steps

Statistical Programming and Linear Regression

Using the built-in statistics library

Average of values in a counter

Computing the coefficient of a correlation

Computing regression parameters

Computing an autocorrelation

Confirming that the data is random – the null hypothesis

Locating outliers

Analyzing many variables in one pass

Other Books You May Enjoy

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Computing an autocorrelation

In many cases, events occur in a repeating cycle. If the data correlates with itself, this is called an autocorrelation. With some data, the interval may be obvious because there's some visible external influence, such as seasons or tides. With some data, the interval may be difficult to discern.

If we suspect we have cyclic data, we can leverage the correlation() function from the Computing the coefficient of correlation recipe, earlier in this chapter, to compute an autocorrelation.

Getting ready

The core concept behind autocorrelation is the idea of a correlation through a shift in time, T. The measurement for this is sometimes expressed as : the correlation between x and x with a time shift of T.

Assume we have a handy correlation function, . It compares two sequences of length n, and , and returns the coefficient of correlation between the two sequences: