Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Secret Recipes of the Python Ninja
  • Table Of Contents Toc
  • Feedback & Rating feedback
Secret Recipes of the Python Ninja

Secret Recipes of the Python Ninja

close
close
Secret Recipes of the Python Ninja

Secret Recipes of the Python Ninja

Overview of this book

This book covers the unexplored secrets of Python, delve into its depths, and uncover its mysteries. You’ll unearth secrets related to the implementation of the standard library, by looking at how modules actually work. You’ll understand the implementation of collections, decimals, and fraction modules. If you haven’t used decorators, coroutines, and generator functions much before, as you make your way through the recipes, you’ll learn what you’ve been missing out on. We’ll cover internal special methods in detail, so you understand what they are and how they can be used to improve the engineering decisions you make. Next, you’ll explore the CPython interpreter, which is a treasure trove of secret hacks that not many programmers are aware of. We’ll take you through the depths of the PyPy project, where you’ll come across several exciting ways that you can improve speed and concurrency. Finally, we’ll take time to explore the PEPs of the latest versions to discover some interesting hacks.
Table of Contents (11 chapters)
close
close

Utilizing requirement files and resolving conflicts

As mentioned previously, a requirements file, requirements.txt, can be created to provide a list of packages to install all at once, via pip install -r requirements.txt. The requirements file can specify specific or minimum versions, or simply specify the library name and the latest version will be installed.

It should be noted that files pulled from the requirements file aren't necessarily installed in a particular order. If you require certain packages to be installed prior to others, you will have to take measures to ensure that the installation is sequential, such as having multiple pip install calls.

Requirements files can specify version numbers of packages explicitly. For example, two different modules (m1 and m2) both depend on a third module (m3). The module m1 requires m3 to be at least version 1.5, but m2 requires it to be no later than version 2.0; the current version of m3 is 2.3. In addition, the latest version of m2 (version 1.7) is known to contain a bug.

Hash digests can be used in requirements files to verify downloaded packages to guard against a compromise of the PyPI database or the HTTPS certificate chain. This is actually a good thing, as in 2017 ten Python libraries (https://www.bleepingcomputer.com/news/security/ten-malicious-libraries-found-on-pypi-python-package-index/) uploaded to PyPI were found to be hosting malicious files.

Because PyPI does not perform any security checks or code auditing when packages are uploaded, it is actually very easy to upload malicious software.

How to do it...

  1. Manually create requirements.txt by typing in the packages to include in the project. The following is an example from https://pip.pypa.io/en/latest/reference/pip_install/#requirements-file-format:

  1. Alternatively, run pip freeze > requirements.txt. This automatically directs the currently installed packages to a properly formatted requirements file.
  2. To implement hash-checking mode, simply include the digest with the package name in the requirements file, demonstrated below:
      FooProject == 1.2 --hash=sha256:<hash_digest>
Note: Supported hash algorithms include: md5, sha1, sha224, sha384, sha256, and sha512.
  1. If there are module conflicts, or special versioning is needed, provide the first module required:
      m1
  1. Indicate the second module, but ensure the version installed pre-dates the known bad version:
      m2<1.7
  1. Provide the third module, ensuring it is at least equal to the minimum version required, but no greater than the maximum version that can be used:
     m3>=1.5, <=2.0

While the preceding screenshot shows some version specifier requirements, here is an example showing some of the different ways to specify module versions in requirements.txt:

        flask
flask-pretty == 0.2.0
flask-security <= 3.0
flask-oauthlib >= 0.9.0, <= 0.9.4

How it works...

In this example, module m1 is specified as a requirement, but the version number doesn't matter; in this case, pip will install the latest version. However, because of the bug in the latest version of m2, an earlier version is specified to be installed. Finally, m3 must be a version between 1.5 and 2.0 to satisfy the installation. Naturally, if one of these conditions can't be met, the installation will fail and the offending library and version numbers will be displayed for further troubleshooting.

There's more...

It's worth noting that pip doesn't have true dependency resolution; it will simply install the first file specified. Thus, it is possible to have dependency conflicts or a sub-dependency that doesn't match the actual requirement. This is why a requirements file is useful, as it alleviates some dependency problems.

Verifying hashes also ensures that a package can't be changed without its version number changing as well, such as in an automated server deployment. This is an ideal situation for efficiency, as it eliminates the need for a private index server that maintains only approved packages.

Visually different images
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Secret Recipes of the Python Ninja
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon