Book Image

Modern Python Standard Library Cookbook

By : Alessandro Molina
Book Image

Modern Python Standard Library Cookbook

By: Alessandro Molina

Overview of this book

The Python 3 Standard Library is a vast array of modules that you can use for developing various kinds of applications. It contains an exhaustive list of libraries, and this book will help you choose the best one to address specific programming problems in Python. The Modern Python Standard Library Cookbook begins with recipes on containers and data structures and guides you in performing effective text management in Python. You will find Python recipes for command-line operations, networking, filesystems and directories, and concurrent execution. You will learn about Python security essentials in Python and get to grips with various development tools for debugging, benchmarking, inspection, error reporting, and tracing. The book includes recipes to help you create graphical user interfaces for your application. You will learn to work with multimedia components and perform mathematical operations on date and time. The recipes will also show you how to deploy different searching and sorting algorithms on your data. By the end of the book, you will have acquired the skills needed to write clean code in Python and develop applications that meet your needs.
Table of Contents (21 chapters)
Title Page
Copyright and Credits
Packt Upsell
Contributors
Preface
Index

Memory and disk buffer


Sometimes, we need to keep certain data in a buffer, such as a file we downloaded from the internet or some data we are generating on the fly.

As the size of such data is not always predictable, is usually not a good idea to keep it all in memory.

If you are downloading a big 32 GB file from the internet that you need to process (such as decompress or parse), it will probably exhaust all your memory if you try to store it into a string before processing it.

That's why it's usually a good idea to rely on tempfile.SpooledTemporaryFile, which will keep the content in memory until it reaches its maximum size and will then move it to a temporary file if it's bigger than the maximum allowed size.

That way, we can have the benefit of keeping an in-memory buffer of our data, without the risk of exhausting all the memory, because as soon as the content is too big, it will be moved to disk.

How to do it...

Like the other tempfile object, creating SpooledTemporaryFile is enough to...