Book Image

Secret Recipes of the Python Ninja

Book Image

Secret Recipes of the Python Ninja

Overview of this book

This book covers the unexplored secrets of Python, delve into its depths, and uncover its mysteries. You’ll unearth secrets related to the implementation of the standard library, by looking at how modules actually work. You’ll understand the implementation of collections, decimals, and fraction modules. If you haven’t used decorators, coroutines, and generator functions much before, as you make your way through the recipes, you’ll learn what you’ve been missing out on. We’ll cover internal special methods in detail, so you understand what they are and how they can be used to improve the engineering decisions you make. Next, you’ll explore the CPython interpreter, which is a treasure trove of secret hacks that not many programmers are aware of. We’ll take you through the depths of the PyPy project, where you’ll come across several exciting ways that you can improve speed and concurrency. Finally, we’ll take time to explore the PEPs of the latest versions to discover some interesting hacks.
Table of Contents (17 chapters)
Title Page
Copyright and Credits
Packt Upsell
Foreword
Contributors
Preface
Index

Project packaging


Everything we have talked about so far is just the basics required to get your project configured and set up for packaging; we haven't actually packaged it yet. To actually create a package that can be installed from PyPI or another package index, you need to run the setup.py script.

How to do it...

  1. Create a source code-based distribution. The minimum required for a package is a source distribution. A source distribution provides the metadata and essential source code files needed by pip for installation. A source distribution is essentially raw code and requires a build step prior to installation to build out the installation metadata from setup.py. A source distribution is created by running python setup.py sdist.
  2. While source distributions are a necessity, it is more convenient to create wheels. Wheel packages are highly recommended, as they are pre-built packages that can be installed without waiting for the build process. This means installation is significantly faster compared to working with a source distribution. There are several types of wheels, depending on whether the project is pure Python and whether it natively supports both Python 2 and 3. To build wheels, you must first install the wheel package: pip install wheel.
  3. The preferred wheel package is a universal wheel. Universal wheels are pure Python, that is, do not contain C-code compiled extensions, and natively support both Python 2 and 3 environments. Universal wheels can be installed anywhere using pip. To build a universal wheel, the following command is used:
      python setup.py bdist_wheel --universal

--universal should only be used when there are no C extensions in use and the Python code runs on both Python 2 and Python 3 without needing modifications, such as running 2to3.bdist_wheel signifies that the distribution is a binary one, as opposed to a source distribution. When used in conjunction with --universal, it does not check to ensure that it is being used correctly, so no warnings will be provided if the criteria are not met. The reason universal wheels shouldn't be used with C extensions is because pip prefers wheels over source distributions. Since an incorrect wheel will mostly likely prevent the C extension from being built, the extension won't be available for use.

  1. Alternatively, pure Python wheels can be used. Pure Python wheels are created when the Python source code doesn't natively support both Python 2 and 3 functionality. If the code can be modified for use between the two versions, such as via 2to3, you can manually create wheels for each version. To build a wheel, use the following command:
      python setup.py bdist_wheel

bdist_wheel will identify the code and build a wheel that is compatible for any Python installation with the same major version number, that is, 2.x or 3.x.

  1. Finally, platform wheels can be used when making packages for specific platforms. Platform wheels are binary builds specific to a certain platform/architecture due to the inclusion of compiled C extensions. Thus, if you need to make a program that is only used on macOS, a platform wheel must be used. The same command as a pure Python wheel is used, but bdist_wheel will detect that the code is not pure Python code and will build a wheel whose name will identify it as only usable on a specific platform. This is the same tag as referenced in the Installing from Wheels section.