Welcome to the world of network programming with Python. Python is a full-featured object-oriented programming language with a standard library that includes everything needed to rapidly build powerful network applications. In addition, it has a multitude of third-party libraries and packages that extend Python to every sphere of network programming. Combined with the fun of using Python, with this book, we hope to get you started on your journey so that you master these tools and produce some great networking code.
In this book, we are squarely targeting Python 3. Although Python 3 is still establishing itself as the successor to Python 2, version 3 is the future of the language, and we want to demonstrate that it is ready for network programming prime time. It offers many improvements over the previous version, many of which improve the network programming experience, with enhanced standard library modules and new additions.
We hope you enjoy this introduction to network programming with Python.
Chapter 1, Network Programming and Python, introduces core networking concepts for readers that are new to networking, and also covers how network programming is approached in Python.
Chapter 2, HTTP and Working with the Web, introduces you to the HTTP protocol and covers how we can retrieve and manipulate web content using Python as an HTTP client. We also take a look at the standard library urllib
and third-party Requests
modules.
Chapter 3, APIs in Action, introduces you to working with web APIs using HTTP. We also cover the XML and JSON data formats, and walk you through developing applications using the Amazon Web Services Simple Storage Service (S3) and Twitter APIs.
Chapter 4, Engaging with E-mails, covers the principle protocols used in sending and receiving e-mails, such as SMTP, POP3, and IMAP, and how to work with them in Python 3.
Chapter 5, Interacting with Remote Systems, guides you through the ways of using Python to connect to servers and performing common administrative tasks, including the execution of shell commands through SSH, file transfers with FTP and SMB, authentication with LDAP, and to monitor systems with SNMP.
Chapter 6, IP and DNS, discusses the details of the Internet Protocol (IP), ways of working with IP in Python, and how to use DNS to resolve hostnames.
Chapter 7, Programming with Sockets, covers using TCP and UDP sockets from Python for writing low-level network applications. We also cover HTTPS and TLS for secure data transport.
Chapter 8, Client and Server Applications, looks at writing client and server programs for socket-based communication. By writing an echo application and a chat application we look at developing basic protocols, framing network data, and compare the multithreading and event-based server architectures.
Chapter 9, Applications for the Web, introduces you to writing web applications in Python. We cover the main approaches, methods of hosting Python web applications, and develop an example application in the Flask microframework.
Appendix, Working with Wireshark, covers packet sniffers, the installation of Wireshark, and how to capture and filter packets using the Wireshark application.
This book is aimed at Python 3. While many of the examples will work in Python 2, you'll get the best experience working through this book with a recent version of Python 3. At the time of writing, the latest version is 3.4.3, and the examples were tested against this.
Though Python 3.4 is the preferred version, all the examples should run on Python 3.1 or later, except for the following:
We're also targeting the Linux operating system, and the assumption is made that you are working on a Linux OS. The examples have been tested on Windows though, and we'll make a note of where there may be differences in the requirements or outcomes.
It is highly recommended that you use Python virtual environments, or "venvs", when you work with this book, and in fact, when doing any work with Python. A venv is an isolated copy of the Python executable and associated files, which provides a separate environment for installing Python modules, independent from the system Python installation. You can have as many venvs as you need, which means that you can have multiple module configurations set up, and you can switch between them easily.
From version 3.3, Python includes a venv
module, which provides this functionality. The documentation and examples are available at https://docs.python.org/3/using/scripts.html. There is also a standalone tool available for earlier versions, which can be found at https://virtualenv.pypa.io/en/latest/.
Most major Linux distributions come preinstalled with Python 2. When installing Python 3 on such a system, it is important to note that we're not replacing the installation of Python 2. Many distributions use Python 2 for core system operations, and these will be tuned for the major version of the system Python. Replacing the system Python can have severe consequences for the running of the OS. Instead, when we install Python 3, it is installed side by side with Python 2. After installing Python 3, it is invoked using the python3.x
executable, where x
is replaced with the corresponding installed minor version. Most packages also provide a symlink
to this executable called python3
, which can be run instead.
Packages to install Python 3.4 are available for most recent distributions, we'll go through the major ones here. If packages are not available, there are still some options that you can use to install a working Python 3.4 environment.
Ubuntu 15.04 and 14.04 come with Python 3.4 already installed; so if you're running these versions, you're already good to go. Note that there is a bug in 14.04, which means pip must be installed manually in any venvs created using the bundled venv
module. You can find information on working around this at http://askubuntu.com/questions/488529/pyvenv-3-4-error-returned-non-zero-exit-status-1.
For earlier versions of Ubuntu, Felix Krull maintains a repository of up-to-date Python installations for Ubuntu. The complete details can be found at https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes.
On Debian, Jessie has a Python 3.4 package (python3.4
), which can be installed directly with apt-get
. Wheezy has a package for 3.2 (python3.2
), and Squeeze has python3.1
, which can be installed similarly. In order to get working Python 3.4 installations on these latter two, it's easiest to use Felix Krull's repositories for Ubuntu.
These distributions don't provide up-to-date Python 3 packages, so we need to use a third-party repository. For Red Hat Enterprise Linux, CentOS, and Scientific Linux, Python 3 can be obtained from the community supported Software Collections (SCL) repository. Instructions on using this repository can be found at https://www.softwarecollections.org/en/scls/rhscl/python33/. At the time of writing, Python 3.3 is the latest available version.
Python 3.4 is available from another repository, the IUS Community repository, sponsored by Rackspace. Instructions on the installation can be found at https://iuscommunity.org/pages/IUSClientUsageGuide.html.
If you're working on a system, which isn't one of the systems mentioned earlier, and you can't find packages for your system to install an up-to-date Python 3, there are still other ways of getting it installed. We'll discuss two methods, Pythonz
and JuJu
.
Pythonz is a program that manages the compilation of Python interpreters from source code. It downloads and compiles Python from source and installs the compiled Python interpreters in your home directory. These binaries can then be used to create venvs. The only limitation with this installation method is that you need a build environment (that is, a C compiler and supporting packages) installed on your system, and dependencies to compile Python. If this doesn't come with your distribution, you will need root access to install this initially. The complete instructions can be found at https://github.com/saghul/pythonz.
JuJu can be used as a last resort, it allows a working Python 3.4 installation on any system without needing root access. It works by creating a tiny Arch Linux installation in a folder, in your home folder and provides tools that allow us to switch to this installation and run commands in it. Using this, we can install Arch's Python 3.4 package, and you can run Python programs using this. The Arch environment even shares your home folder with your system, so sharing files between environments is easy. The JuJu home page is available at https://github.com/fsquillace/juju.
JuJu should work on any distribution. To install it we need to do this:
$ mkdir ~/.juju $ curl https:// bitbucket.org/fsquillace/juju-repo/raw/master/juju- x86_64.tar.gz | tar -xz -C ~/.juju
This downloads and extracts the JuJu image to ~/.juju
. You'll need to replace the x86_64
with x86
if you're running on a 32-bit system. Next, set up PATH
to pick up the JuJu commands:
$ export PATH=~/.juju/opt/juju/bin:$PATH
It's a good idea to add this to your .bashrc
, so you don't need to run it every time you log in. Next, we install Python in the JuJu
environment, we only need to do this once:
$ juju -f $ pacman --sync refresh $ pacman --sync --sysupgrade $ pacman --sync python3 $ exit
These commands first activate the JuJu
environment as root, then use the pacman
Arch Linux package manager to update the system and install Python 3.4. The final exit
command exits the JuJu
environment. Finally, we can access the JuJu
environment as a regular user:
$ juju
We can then start using the installed Python 3:
$ python3 Python 3.4.3 (default, Apr 28 2015, 19:59:08) [GCC 4.7.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
Compared to some of the older Linux distributions, installing Python 3.4 on Windows is relatively easy; just download the Python 3.4 installer from http://www.python.org and run it. The only hitch is that it requires administrator privileges to do so, so if you're on a locked down machine, things are trickier. The best solution at the moment is WinPython, which is available at http://winpython.github.io.
We assume that you have a working Internet connection. Several chapters use Internet resources extensively, and there is no real way to emulate these offline. Having a second computer is also useful to explore some networking concepts, and for trying out network applications across a real network.
We also use the Wireshark packet sniffer in several chapters. This will require a machine where you have root access (or administrator access in Windows). Wireshark installers and installation instructions are available at https://www.wireshark.org. An introduction to using Wireshark can be found in the Appendix, Working with Wireshark.
If you're a Python developer, or system administrator with Python experience, and you're looking forward to take your first step in network programming, then this book is for you. Whether you're working with networks for the first time or looking to enhance your existing networking and Python skills, you will find this book very useful.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "IP addresses have been assigned to your computer by running the ip addr
or ipconfig /all
command on Windows."
A block of code is set as follows:
import sys, urllib.request try: rfc_number = int(sys.argv[1]) except (IndexError, ValueError): print('Must supply an RFC number as first argument') sys.exit(2) template = 'http://www.ietf.org/rfc/rfc{}.txt' url = template.format(rfc_number) rfc_raw = urllib.request.urlopen(url).read() rfc = rfc_raw.decode() print(rfc)
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are highlighted:
<body>
...
<div id="content">
<h1>Debian “jessie” Release Information</h1>
<p>Debian 8.0 was
released October 18th, 2014.
The release included many major
changes, described in
...
Any command-line input or output is written as follows:
$ python RFC_downloader.py 2324 | less
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "We can see there's a list of interfaces below the Start button."
We follow PEP 8 as closely as we can, but we also follow the principle that practicality beats purity, and do deviate in a few areas. Imports are often performed on a single line to save space, and we may not strictly adhere to wrapping conventions do to the nature of printed media; we aim for "readability counts".
We have also chosen to focus on the procedural programming style rather than use object-oriented examples. The reason for this is that it is generally easier for someone familiar with object oriented programming to rework procedural examples into an object oriented format than it is for someone unfamiliar with OOP to do the reverse.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>
, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]>
with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>
, and we will do our best to address the problem.