Book Image

Learning Python Web Penetration Testing

By : Christian Martorella
Book Image

Learning Python Web Penetration Testing

By: Christian Martorella

Overview of this book

Web penetration testing is the use of tools and code to attack a website or web app in order to assess its vulnerability to external threats. While there are an increasing number of sophisticated, ready-made tools to scan systems for vulnerabilities, the use of Python allows you to write system-specific scripts, or alter and extend existing testing tools to find, exploit, and record as many security weaknesses as possible. Learning Python Web Penetration Testing will walk you through the web application penetration testing methodology, showing you how to write your own tools with Python for each activity throughout the process. The book begins by emphasizing the importance of knowing how to write your own tools with Python for web application penetration testing. You will then learn to interact with a web application using Python, understand the anatomy of an HTTP request, URL, headers and message body, and later create a script to perform a request, and interpret the response and its headers. As you make your way through the book, you will write a web crawler using Python and the Scrappy library. The book will also help you to develop a tool to perform brute force attacks in different parts of the web application. You will then discover more on detecting and exploiting SQL injection vulnerabilities. By the end of this book, you will have successfully created an HTTP proxy based on the mitmproxy tool.
Table of Contents (9 chapters)

Understanding the web application penetration testing process

In this section, we will understand what web application penetration testing is and the process behind it. We will start by learning what web application penetration testing is, the importance of performing these tests, what professional methodologies look like, and we'll briefly explain why it is important to have skills to use Python to write our own tools.

Penetration testing is a type of security testing that evaluates the security of an application from the perspective of an attacker. It is an offensive exercise where you have to think like an attacker and understand the developers as well as the technology involved in order to unveil all the flaws.

The goal is to identify all the flaws and demonstrate how they can be exploited by an attacker, and what the impact will be on our company. Finally, the report will provide solutions to fix the issues that have been detected. It's a manual and dynamic test. Manual means that it heavily depends on the knowledge of the person doing the test, and that is why learning how to write your own penetration testing tools is important, and will give you an edge in your career. Dynamic testing is where we test the running application. It is not a static analysis of the source code. The security test is useful to validate and verify the effect of the application security controls to us and to identify the lax of these security controls.

So, why should we perform penetration testing? Nowadays, IT has taken the world by storm. Most of the company processes and data are handled by computers. This is the reason why companies need to invest in security testing, in order to validate the effectiveness of security controls, and many a times the lack of them.

One report by EMC (https://www.scmagazine.com/study-it-leaders-count-the-cost-of-breaches-data-loss-and-downtime/article/542793/) states that the average report regarding annual financial loss per company is 497,037 USD for down time, 860,273 USD for security breaches, and 585,892 USD for data loss. Plus, all the time, the company resources are put into incident response and fixing, testing, and deploying the issue:

That is why performing penetration testing will help companies to protect their customer's data, intellectual property, and services. Penetration testing is a simple methodology formed by four main sections, which are as follows:

  • Reconnaissance: In this phase, we'll gather information to identify the technologies used, the infrastructure supporting the application, software configuration, load balances, and so on. This phase is also known as fingerprinting.
  • Mapping: We then move into the mapping phase, where we build a map or diagram of the application pages and functionalities. We aim to identify the components and their relationships. One of the techniques to support mapping is spidering or crawling. Also, in this phase, we'll discover nonlinked resources by performing brute force attacks.
  • Vulnerability: Once we have all the components, parameters, forms, and functionalities mapped out, we move to phase three, where we'll start vulnerability discovery.
  • Exploitation: After identifying all the vulnerabilities, we can move to the last phase, which is the exploitation of the vulnerabilities. Depending on the scope of the pen test, once you exploit vulnerability, you can start the process all over again from your new vantage point. Usually, this the target DMZ, which you would try to get into their internal network segment.

One step that is not represented here is the reporting phase, where you document all the findings so that you can present them to your customer, company.

Finally, there are two types of penetration tests, which are the black box and the white box. Black box test takes place when you don't have any information about the target, which is basically the same situation as an attacker, and white box test takes place when the customer provides us with documentation, source code, and configurations to accelerate the process, and we only focus on interesting areas.

You maybe wondering, what areas should you test during this process? These are some of the most important ones to cover:

  • Configuration and deployment management testing
  • Identity management testing
  • Authentication testing
  • Authorization testing
  • Session management testing
  • Input validation
  • Testing error handling
  • Cryptography
  • Business logic testing
  • Client-side testing

We'll cover some of these areas in this chapter.

You can expand your knowledge on these areas by reading the OWASP testing guide: https://www.owasp.org/index.php/OWASP_Testing_Project.

So, why build your own tools? Web applications are very different since they're developed using multiple technologies, combinations, flows, and implementations.

This is the reason why there is not a single tool that will cover all the scenarios that you will find during your career. Many times, we'll write scripts to test specific issues or to make certain tasks, and to exploit a vulnerability. During the course of this book, we'll see how to write tools and test different areas such as authentication, input validation, and discovery, and we'll end up writing a simple Hypertext Transfer Protocol (HTTP) proxy that could be the foundation of our own security scanner. Writing your own tools is a valuable skill that will put you ahead of many penetration testers that do not have the capability to adapt tools, or write their own. In certain penetration test engagements, this could make all the difference.