Book Image

Learning Python Web Penetration Testing

By : Christian Martorella
Book Image

Learning Python Web Penetration Testing

By: Christian Martorella

Overview of this book

Web penetration testing is the use of tools and code to attack a website or web app in order to assess its vulnerability to external threats. While there are an increasing number of sophisticated, ready-made tools to scan systems for vulnerabilities, the use of Python allows you to write system-specific scripts, or alter and extend existing testing tools to find, exploit, and record as many security weaknesses as possible. Learning Python Web Penetration Testing will walk you through the web application penetration testing methodology, showing you how to write your own tools with Python for each activity throughout the process. The book begins by emphasizing the importance of knowing how to write your own tools with Python for web application penetration testing. You will then learn to interact with a web application using Python, understand the anatomy of an HTTP request, URL, headers and message body, and later create a script to perform a request, and interpret the response and its headers. As you make your way through the book, you will write a web crawler using Python and the Scrappy library. The book will also help you to develop a tool to perform brute force attacks in different parts of the web application. You will then discover more on detecting and exploiting SQL injection vulnerabilities. By the end of this book, you will have successfully created an HTTP proxy based on the mitmproxy tool.
Table of Contents (9 chapters)

Making our crawler recursive

In this section, we'll start learning how to extract links, and then we'll use them to make the crawler recursive. Now that we have created the basic structure of a crawler, let's add some functionality:

  1. First, let's copy the prepared spiderman.py file for this exercise. Copy it from examples/spiders/spiderman-recursive.py to basic_crawler/basic_crawler/spiders/spiderman.py.
  1. Then, go back to our editor. As we would like to make the crawler recursive, for this purpose, we will once again work the spiderman.py file and start with adding another extractor. However, this time we'll add the links instead of titles, as highlighted in the following screenshot:
  1. Also, we need to make sure that the links are valid and complete, so we'll create a regular expression that will validate links highlighted in the following screenshot...