Book Image

Learning Python Networking - Second Edition

By : José Manuel Ortega, Dr. M. O. Faruque Sarker, Sam Washington
Book Image

Learning Python Networking - Second Edition

By: José Manuel Ortega, Dr. M. O. Faruque Sarker, Sam Washington

Overview of this book

Network programming has always been a demanding task. With full-featured and well-documented libraries all the way up the stack, Python makes network programming the enjoyable experience it should be. Starting with a walk through of today's major networking protocols, through this book, you'll learn how to employ Python for network programming, how to request and retrieve web resources, and how to extract data in major formats over the web. You will utilize Python for emailing using different protocols, and you'll interact with remote systems and IP and DNS networking. You will cover the connection of networking devices and configuration using Python 3.7, along with cloud-based network management tasks using Python. As the book progresses, socket programming will be covered, followed by how to design servers, and the pros and cons of multithreaded and event-driven architectures. You'll develop practical clientside applications, including web API clients, email clients, SSH, and FTP. These applications will also be implemented through existing web application frameworks.
Table of Contents (19 chapters)
Free Chapter
1
Section 1: Introduction to Network and HTTP Programming
4
Section 2: Interacting with APIs, Web Scraping, and Server Scripting
9
Section 3: IP Address Manipulation and Network Automation
13
Section 4: Sockets and Server Programming

Working with Scrapy in the cloud

In this section, we will explore Scrapy for deploying spiders and crawlers in the cloud.

Scrapinghub

The first step is register in the Scrapinghub service, which can be done at the following URL: https://app.scrapinghub.com/account/login/.

Scrapy Cloud is a platform for running web crawlers and spiders, where spiders are executing in cloud servers and scale on demand: https://scrapinghub.com/scrapy-cloud.

To deploy projects into Scrapy Cloud, you will need the Scrapinghub command-line client, called shub, and it can be installed with the pip install command. You can check if you have the latest version:

$ pip install shub --upgrade

The next step is to create a project in Scrapinghub and deploy...