This book will introduce you to more advanced features of the Python programming language. The focus is on creating the highest quality Python programs possible. This often means creating programs that have the highest performance or are the most maintainable. This means exploring design alternatives and determining which design offers the best performance while still being a good fit with the problem that is being solved.
Most of the book will look at a number of alternatives for a given design. Some will have better performance. Some will seem simpler or be a better solution for the problem domain. It's essential to locate the best algorithms and optimal data structures to create the most value with the least computer processing. Time is money, and programs that save time will create more value for their users.
Python makes a number of internal features directly available to our application programs. This means that our programs can be very tightly integrated with existing Python features. We can leverage numerous Python features by ensuring that our OO designs integrate well.
We'll often focus on a specific problem and examine several variant solutions to the problem. As we look at different algorithms and data structures, we'll see different memory and performance alternatives. It's an important OO design skill to work through alternate solutions in order to properly optimize the final application.
One of the more important themes of this book is that there's no single best approach to any problem. There are a number of alternative approaches with different attributes.
On programming style, the subject of style generates a surprising amount of interest. The astute reader will note that the examples do not meticulously conform to PEP-8 in every single particular detail of the name choice or punctuation.
As we move towards achieving mastery over object-oriented Python, we'll spend a great deal of time reading Python code from a variety of sources. We'll observe wide variability even within the Python Standard Library modules. Rather than presenting examples that are all perfectly consistent, we've opted for some inconsistency, the lack of consistency will better confirm with code as seen in the various open source projects encountered in the wild.
We'll cover three advanced Python topics in a series of chapters that dig into the details.
Some Preliminaries, covers some preliminary topics, such as unittest, doctest, docstrings, and some special method names.
Part 1, Pythonic Classes via Special Methods: This part looks more deeply at object-oriented programming techniques and how we can more tightly integrate the class definitions of our applications with Python's built-in features. It consists of nine chapters, which are as follows:
Chapter 1, The _init_() Method, provides us with a detailed description and implementation of the
_init_()method. We will look at different forms of initialization for simple objects. From this, we can look into more complex objects that involve collections and containers.Chapter 2, Integrating Seamlessly with Python – Basic Special Methods, will explain in detail as to how we can expand a simple class definition to add special methods. We'll need to take a look at the default behavior inherited from the object so that we can understand what overrides are needed and when they're actually needed.
Chapter 3, Attribute Access, Properties, and Descriptors, shows us how the default processing works in some detail. We need to decide where and when to override the default behavior. We will also explore descriptors and gain a much deeper understanding on how Python's internals work.
Chapter 4, The ABCs of Consistent Design, looks at the abstract base classes in the
collections.abcmodule in general. We'll look at the general concepts behind the various containers and collections that we might want to revise or extend. Similarly, we'll look at the concepts behind the numbers that we might want to implement.Chapter 5, Using Callables and Contexts, looks at several ways to create context managers using the tools in
contextlib. We'll show you a number of variant designs for callable objects. This will show you why a stateful callable object is sometimes more useful than a simple function. We'll also take a look at how to use some of the existing Python context managers before we dive in and write our own context manager.Chapter 6, Creating Containers and Collections, focuses on the basics of container classes. We'll review the variety of special methods that are involved in being a container and offering the various features that containers offer. We'll address extending built-in containers to add features. We'll also look at wrapping built-in containers and delegating methods through the wrapper to the underlying container.
Chapter 7, Creating Numbers, covers these essential arithmetic operators:
+,-,*,/,//,%, and**. We'll also take a look at these comparison operators:<,>,<=,>=,==, and!=. We'll finish by summarizing some of the design considerations that go into extending or creating new numbers.Chapter 8, Decorators and Mixins – Cross-cutting Aspects, covers simple function decorators, function decorators with arguments, class decorators, and method decorators.
Part 2, Persistence and Serialization: A persistent object has been serialized to a storage medium. Perhaps it's transformed to JSON and written to the filesystem. An ORM layer can store the object in a database. This part will take a look at the alternatives to handle persistence. This section contains five chapters, which are as follows:
Chapter 9, Serializing and Saving – JSON, YAML, Pickle, CSV, and XML, covers simple persistence using libraries focused on various data representations such as JSON, YAML, pickle, XML, and CSV.
Chapter 10, Storing and Retrieving Objects via Shelve, explains basic database operations with Python modules, such as
shelve(anddbm).Chapter 11, Storing and Retrieving Objects via SQLite, moves to the more complex world of SQL and the relational database. Because SQL features don't match object-oriented programming features well, we have an impedance mismatch problem. A common solution is to use ORM to allow us to persist a large domain of objects.
Chapter 12, Transmitting and Sharing Objects, takes a look at the HTTP protocol, JSON, YAML, and XML representation to transmit an object.
Chapter 13, Configuration Files and Persistence, covers various ways in which a Python application can work with a configuration file.
Part 3, Testing, Debugging, Deploying, and Maintaining: We'll show you how to gather data to support and debug high-performance programs. This will include information on creating the best possible documentation in order to reduce the confusion and complexity of the support. This section contains the final five chapters, which are as follows:
Chapter 14, The Logging and Warning Modules, takes a look at using the
loggingandwarningmodules to create audit information, as well as debug. We'll take a significant step beyond using theprint()function.Chapter 15, Designing for Testability, covers designing for testability and how we use
unittestanddoctest.Chapter 16, Coping with the Command Line, takes a look at using the
argparsemodule to parse options and arguments. We'll take this a step further and use the command design pattern to create program components that can be combined and expanded without resorting to writing shell scripts.Chapter 17, The Module and Package Design, covers module and package design. This is a higher-level set of considerations. We will take a look at related classes in a module and related modules in a package.
Chapter 18, Quality and Documentation, covers how we can document our design to create trust that our software is correct and has been properly implemented.
In order to compile and run the examples mentioned in this book, you require the following software:
Python Version 3.2 or higher with the standard suite of libraries. We'll focus on Python 3.3, but the differences from 3.2 are minor.
We'll take a look at some additional packages. These include PyYaml, SQLAlchemy, and Jinja2.
http://www.sqlalchemy.org. When building this, check the installation guide, http://docs.sqlalchemy.org/en/rel_0_9/intro.html#installation. Using the
--without-cextensionsoption can simplify installation.
Optionally, you might want to add Sphinx or Docutils to your environment, as we'll cover them as well.
This is advanced Python. You'll need to be quite familiar with Python 3. You'll also benefit from having fairly large or complex problems to solve.
If you are a skilled programmer with the other languages, you may find this book helpful if you want to switch to Python. This book doesn't introduce syntax or other foundational concepts.
Advanced Python 2 programmers may find this helpful when they switch to Python 3. We won't cover any of the conversion utilities (such as from Version 2 to 3) or any of the coexistence libraries (such as six.) This book is focused on new development that has happened entirely in Python 3.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text are shown as follows: "We can access other Python modules through the use of the import statement."
A block of code is set as follows:
class Friend(Contact):
def __init__(self, name, email, phone):
self.name = name
self.email = email
self.phone = phoneWhen we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
class Friend(Contact):
def __init__(self, name, email, phone):
self.name = name
self.email = email
self.phone = phoneAny command-line input or output is written as follows:
>>> e = EmailableContact("John Smith", "[email protected]") >>> Contact.all_contacts
New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "We use this feature to update the label to a new random value every time we click on the Roll! button".
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or e-mail <[email protected]>.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub. com/support and register to have the files e-mailed directly to you.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the let us know link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.