Book Image

Learning Python Network Programming

By : Dr. M. O. Faruque Sarker, Samuel B Washington, Sam Washington
Book Image

Learning Python Network Programming

By: Dr. M. O. Faruque Sarker, Samuel B Washington, Sam Washington

Overview of this book

Table of Contents (17 chapters)
Learning Python Network Programming
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Preface

Welcome to the world of network programming with Python. Python is a full-featured object-oriented programming language with a standard library that includes everything needed to rapidly build powerful network applications. In addition, it has a multitude of third-party libraries and packages that extend Python to every sphere of network programming. Combined with the fun of using Python, with this book, we hope to get you started on your journey so that you master these tools and produce some great networking code.

In this book, we are squarely targeting Python 3. Although Python 3 is still establishing itself as the successor to Python 2, version 3 is the future of the language, and we want to demonstrate that it is ready for network programming prime time. It offers many improvements over the previous version, many of which improve the network programming experience, with enhanced standard library modules and new additions.

We hope you enjoy this introduction to network programming with Python.

What this book covers

Chapter 1, Network Programming and Python, introduces core networking concepts for readers that are new to networking, and also covers how network programming is approached in Python.

Chapter 2, HTTP and Working with the Web, introduces you to the HTTP protocol and covers how we can retrieve and manipulate web content using Python as an HTTP client. We also take a look at the standard library urllib and third-party Requests modules.

Chapter 3, APIs in Action, introduces you to working with web APIs using HTTP. We also cover the XML and JSON data formats, and walk you through developing applications using the Amazon Web Services Simple Storage Service (S3) and Twitter APIs.

Chapter 4, Engaging with E-mails, covers the principle protocols used in sending and receiving e-mails, such as SMTP, POP3, and IMAP, and how to work with them in Python 3.

Chapter 5, Interacting with Remote Systems, guides you through the ways of using Python to connect to servers and performing common administrative tasks, including the execution of shell commands through SSH, file transfers with FTP and SMB, authentication with LDAP, and to monitor systems with SNMP.

Chapter 6, IP and DNS, discusses the details of the Internet Protocol (IP), ways of working with IP in Python, and how to use DNS to resolve hostnames.

Chapter 7, Programming with Sockets, covers using TCP and UDP sockets from Python for writing low-level network applications. We also cover HTTPS and TLS for secure data transport.

Chapter 8, Client and Server Applications, looks at writing client and server programs for socket-based communication. By writing an echo application and a chat application we look at developing basic protocols, framing network data, and compare the multithreading and event-based server architectures.

Chapter 9, Applications for the Web, introduces you to writing web applications in Python. We cover the main approaches, methods of hosting Python web applications, and develop an example application in the Flask microframework.

Appendix, Working with Wireshark, covers packet sniffers, the installation of Wireshark, and how to capture and filter packets using the Wireshark application.

What you need for this book

This book is aimed at Python 3. While many of the examples will work in Python 2, you'll get the best experience working through this book with a recent version of Python 3. At the time of writing, the latest version is 3.4.3, and the examples were tested against this.

Though Python 3.4 is the preferred version, all the examples should run on Python 3.1 or later, except for the following:

  • The asyncio example in Chapter 8, Client and Server Applications, as the asyncio module was only included in Version 3.4

  • The Flask example in Chapter 9, Applications for the Web, which requires Python 3.3 or later

We're also targeting the Linux operating system, and the assumption is made that you are working on a Linux OS. The examples have been tested on Windows though, and we'll make a note of where there may be differences in the requirements or outcomes.

Virtual environments

It is highly recommended that you use Python virtual environments, or "venvs", when you work with this book, and in fact, when doing any work with Python. A venv is an isolated copy of the Python executable and associated files, which provides a separate environment for installing Python modules, independent from the system Python installation. You can have as many venvs as you need, which means that you can have multiple module configurations set up, and you can switch between them easily.

From version 3.3, Python includes a venv module, which provides this functionality. The documentation and examples are available at https://docs.python.org/3/using/scripts.html. There is also a standalone tool available for earlier versions, which can be found at https://virtualenv.pypa.io/en/latest/.

Installing Python 3

Most major Linux distributions come preinstalled with Python 2. When installing Python 3 on such a system, it is important to note that we're not replacing the installation of Python 2. Many distributions use Python 2 for core system operations, and these will be tuned for the major version of the system Python. Replacing the system Python can have severe consequences for the running of the OS. Instead, when we install Python 3, it is installed side by side with Python 2. After installing Python 3, it is invoked using the python3.x executable, where x is replaced with the corresponding installed minor version. Most packages also provide a symlink to this executable called python3, which can be run instead.

Packages to install Python 3.4 are available for most recent distributions, we'll go through the major ones here. If packages are not available, there are still some options that you can use to install a working Python 3.4 environment.

Ubuntu and Debian

Ubuntu 15.04 and 14.04 come with Python 3.4 already installed; so if you're running these versions, you're already good to go. Note that there is a bug in 14.04, which means pip must be installed manually in any venvs created using the bundled venv module. You can find information on working around this at http://askubuntu.com/questions/488529/pyvenv-3-4-error-returned-non-zero-exit-status-1.

For earlier versions of Ubuntu, Felix Krull maintains a repository of up-to-date Python installations for Ubuntu. The complete details can be found at https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes.

On Debian, Jessie has a Python 3.4 package (python3.4), which can be installed directly with apt-get. Wheezy has a package for 3.2 (python3.2), and Squeeze has python3.1, which can be installed similarly. In order to get working Python 3.4 installations on these latter two, it's easiest to use Felix Krull's repositories for Ubuntu.

RHEL, CentOS, Scientific Linux

These distributions don't provide up-to-date Python 3 packages, so we need to use a third-party repository. For Red Hat Enterprise Linux, CentOS, and Scientific Linux, Python 3 can be obtained from the community supported Software Collections (SCL) repository. Instructions on using this repository can be found at https://www.softwarecollections.org/en/scls/rhscl/python33/. At the time of writing, Python 3.3 is the latest available version.

Python 3.4 is available from another repository, the IUS Community repository, sponsored by Rackspace. Instructions on the installation can be found at https://iuscommunity.org/pages/IUSClientUsageGuide.html.

Fedora

Fedora 21 and 22 provide Python 3.4 with the python3 package:

$ yum install python3

For earlier versions of Fedora, use the repositories listed in the preceding Red Hat section.

Alternative installation methods

If you're working on a system, which isn't one of the systems mentioned earlier, and you can't find packages for your system to install an up-to-date Python 3, there are still other ways of getting it installed. We'll discuss two methods, Pythonz and JuJu.

Pythonz

Pythonz is a program that manages the compilation of Python interpreters from source code. It downloads and compiles Python from source and installs the compiled Python interpreters in your home directory. These binaries can then be used to create venvs. The only limitation with this installation method is that you need a build environment (that is, a C compiler and supporting packages) installed on your system, and dependencies to compile Python. If this doesn't come with your distribution, you will need root access to install this initially. The complete instructions can be found at https://github.com/saghul/pythonz.

JuJu

JuJu can be used as a last resort, it allows a working Python 3.4 installation on any system without needing root access. It works by creating a tiny Arch Linux installation in a folder, in your home folder and provides tools that allow us to switch to this installation and run commands in it. Using this, we can install Arch's Python 3.4 package, and you can run Python programs using this. The Arch environment even shares your home folder with your system, so sharing files between environments is easy. The JuJu home page is available at https://github.com/fsquillace/juju.

JuJu should work on any distribution. To install it we need to do this:

$ mkdir ~/.juju
$ curl https:// bitbucket.org/fsquillace/juju-repo/raw/master/juju- x86_64.tar.gz | tar -xz -C ~/.juju

This downloads and extracts the JuJu image to ~/.juju. You'll need to replace the x86_64 with x86 if you're running on a 32-bit system. Next, set up PATH to pick up the JuJu commands:

$ export PATH=~/.juju/opt/juju/bin:$PATH

It's a good idea to add this to your .bashrc, so you don't need to run it every time you log in. Next, we install Python in the JuJu environment, we only need to do this once:

$ juju -f
$ pacman --sync refresh
$ pacman --sync --sysupgrade
$ pacman --sync python3
$ exit

These commands first activate the JuJu environment as root, then use the pacman Arch Linux package manager to update the system and install Python 3.4. The final exit command exits the JuJu environment. Finally, we can access the JuJu environment as a regular user:

$ juju

We can then start using the installed Python 3:

$ python3 
Python 3.4.3 (default, Apr 28 2015, 19:59:08)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Windows

Compared to some of the older Linux distributions, installing Python 3.4 on Windows is relatively easy; just download the Python 3.4 installer from http://www.python.org and run it. The only hitch is that it requires administrator privileges to do so, so if you're on a locked down machine, things are trickier. The best solution at the moment is WinPython, which is available at http://winpython.github.io.

Other requirements

We assume that you have a working Internet connection. Several chapters use Internet resources extensively, and there is no real way to emulate these offline. Having a second computer is also useful to explore some networking concepts, and for trying out network applications across a real network.

We also use the Wireshark packet sniffer in several chapters. This will require a machine where you have root access (or administrator access in Windows). Wireshark installers and installation instructions are available at https://www.wireshark.org. An introduction to using Wireshark can be found in the Appendix, Working with Wireshark.

Who this book is for

If you're a Python developer, or system administrator with Python experience, and you're looking forward to take your first step in network programming, then this book is for you. Whether you're working with networks for the first time or looking to enhance your existing networking and Python skills, you will find this book very useful.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "IP addresses have been assigned to your computer by running the ip addr or ipconfig /all command on Windows."

A block of code is set as follows:

import sys, urllib.request

try:
    rfc_number = int(sys.argv[1])
except (IndexError, ValueError):
    print('Must supply an RFC number as first argument')
    sys.exit(2)

template = 'http://www.ietf.org/rfc/rfc{}.txt'
url = template.format(rfc_number)
rfc_raw = urllib.request.urlopen(url).read()
rfc = rfc_raw.decode()
print(rfc)

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are highlighted:

<body>
...
<div id="content">
<h1>Debian &ldquo;jessie&rdquo; Release Information</h1>
<p>Debian 8.0 was
released October 18th, 2014.
The release included many major
changes, described in
...

Any command-line input or output is written as follows:

$ python RFC_downloader.py 2324 | less

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "We can see there's a list of interfaces below the Start button."

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

We follow PEP 8 as closely as we can, but we also follow the principle that practicality beats purity, and do deviate in a few areas. Imports are often performed on a single line to save space, and we may not strictly adhere to wrapping conventions do to the nature of printed media; we aim for "readability counts".

We have also chosen to focus on the procedural programming style rather than use object-oriented examples. The reason for this is that it is generally easier for someone familiar with object oriented programming to rework procedural examples into an object oriented format than it is for someone unfamiliar with OOP to do the reverse.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail , and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at , and we will do our best to address the problem.