There is a huge misconception about what penetration testing is. This is common even among professionals who have recently entered the field. New penetration testers or professionals who request penetration tests often say that these tests prove the exploitability of vulnerabilities, the susceptibility of an environment to exploitation, or just the presence of vulnerabilities. This misunderstanding manifests itself into real impacts on engagements as they are scoped, sourced, and conducted. Further, this mistaken perception includes the thought that a penetration test will find all vulnerabilities, it will be able to find unknown zero days every time, and all objectives will always be met irrespective of the controls put in place.
A penetration test is the practice of assessing an organization's security strategy's ability to protect critical data from the actions of a malicious actor. A security strategy is the organization's overarching information security program. It focuses on maintaining the confidentiality, integrity, and availability of the organization's critical data and resources. This is to mitigate risk to an acceptable level by using a combination of people, processes, and technology. The difference between the first and the second definition of a penetration test is night and day.
The first definition focuses solely on vulnerabilities; this means that people expect the activity that an assessor will perform to be related to exploiting or finding vulnerabilities or simple misconfigurations. It does not take into account bad practices related to the policies, processes, or insecure relationships that the organization may have. These preconceived notions often have the following significant impacts for both organizations and new assessors.
Organizational leadership will not create goals related to breaching access controls related to critical data repositories or identifying critical data locations. There will also be an initial belief that Intrusion Protection Systems (IPS) and Intrusion Detection Systems (IDS) are the linchpin to preventing a compromise; all experienced assessors know that this is not true. Additionally, assessments may not be scoped in a manner that would provide realistic results. The most damaging result of this misunderstanding is that the organization may not be able to identify when an assessor is missing the skills necessary to execute the required engagement.
Note
Similarly, new assessors have the misconception that a Vulnerability Management Solution (VMS) such as Nexpose, Nessus, Qualys, or others will identify the way into an environment. These may highlight ways to get into a system, but there is a high rate of false positives and true negatives. A false positive means something was identified as vulnerable, but it is not. The opposite of a false positive is a true negative, which means that something was identified as secure, but it is instead vulnerable.
If vulnerabilities are not within the database, then the system will not identify the vulnerability that could grant access. VMS will not highlight the chained attacks related to bad practices or processes, which would be classified as a weakness or vulnerability. The use of these tools for penetration tests makes them exceedingly noisy, and they encourage assessors to simulate attacks that are relatively outdated.
Most malicious actors take advantage of the path of least resistance, which usually does not relate to Remote Code Exploits such as the famous MS08-067 or MS06-40. Instead, an assessor should step back and look for insecure associations and configurations that may provide unnoticed access. Most senior assessors do not use VMS tools during penetration tests, but instead focus on assessing environments manually.
Many of these misconceptions relate directly to other types of engagements. This comes from other security assessments being advertised as penetration tests, or from people either running or receiving the results of these engagements. In the following section, a sample of assessments that are often confused with penetration tests is listed. It should be enough to highlight the differences between an actual penetration test and other security assessments and activities.