Many organizations invest substantial amounts of time and cost in designing and implementing various security controls. Some even deploy multi-layered controls following the principle of defense-in-depth. Implementing strong security controls is certainly required; however, it's equally important to test if the controls deployed are indeed working as expected.

For example, an organization may choose to deploy the latest and best in the class firewall to protect its perimeters. The firewall administrator somehow misconfigures the rules. So however good the firewall may be, if it's not configured properly, it's still going to allow bad traffic in. In this case, a thorough testing and/or review of firewall rules would have helped identify and eliminate unwanted rules and retain the required ones.

Whenever a new system is developed, it strictly and vigorously undergoes quality assurance (QA) testing. This is to ensure that the newly developed system is functioning correctly as per the business requirements and specifications. On parallel lines, testing of security controls is also vital to ensure they are functioning as specified. Security tests could be of different types, as discussed in the next section.

Security tests could be categorized in multiple ways based on the context and the purpose they serve. The following diagram shows a high-level classification of the types of security tests:

The primary objective of security tests is to ensure that a control is functioning properly. The tests could be a combination of automated scans, penetration tests using tools, and manual attempts to reveal security flaws. It's important to note that security testing isn't a one-time activity and should be performed at regular intervals. When planning for testing of security controls, the following factors should be considered:

• Resources (hardware, software, and skilled manpower) available for security testing
• Criticality rating for the systems and applications protected by the controls
• The probability of a technical failure of the mechanism implementing the control
• The probability of a misconfiguration of a control that would endanger the security
• Any other changes, upgrades, or modifications in the technical environment that may affect the control performance
• Difficulty and time required for testing a control
• Impact of the test on regular business operations

Only after determining these factors, a comprehensive assessment and testing strategy can be designed and validated. This strategy may include regular automated tests complemented by manual tests. For example, an e-commerce platform may be subjected to automated vulnerability scanning on a weekly basis with immediate alert notifications to administrators when the scan detects a new vulnerability. The automated scan requires intervention from administrators once it's configured and triggered, so it is easy to scan frequently.

The security team may choose to complement automated scans with a manual penetration test performed by an internal or external consultant for a fixed fee. Security tests can be performed on quarterly, bi-annually, or on an annual basis to optimize costs and efforts.

Unfortunately, many security testing programs begin on a haphazard and ad hoc basis by simply pointing fancy new tools at whatever systems are available in the network. Testing programs should be thoughtfully designed and include rigorous, routine testing of systems using a risk-based approach.

Certainly, security tests cannot be termed complete unless the results are carefully reviewed. A tool may produce a lot of false positives which could be eliminated only by manual reviews. The manual review of a security test report also helps in determining the severity of the vulnerability in context to the target environment.

For example, an automated scanning tool may detect cross-site scripting in a publicly hosted e-commerce application as well as in a simple help-and-support intranet portal. In this case, although the vulnerability is the same in both applications, the earlier one carries more risk as it is internet-facing and has many more users than the latter.

Vulnerability assessment and penetration testing are quite often used interchangeably. However, both are different with respect to the purpose they serve. To understand the difference between the two terms, let's consider a real-world example.

There is a bank that is located on the outskirts of a city and in quite a secluded area. There is a gang of robbers who intend to rob this bank. The robbers start planning on how they could execute their plan. Some of them visit the bank dressed as normal customers and note a few things:

• The bank has only one security guard who is unarmed
• The bank has two entrances and three exits
• There are no CCTV cameras installed
• The door to the locker compartment appears to be weak

With these findings, the robbers just did a vulnerability assessment. Now whether or not these vulnerabilities could be exploited in reality to succeed with the robbery plan would become evident only when they actually rob the bank. If they rob the bank and succeed in exploiting the vulnerabilities, they would have achieved penetration testing.

So, in a nutshell, checking whether a system is vulnerable is vulnerability assessment, whereas actually exploiting the vulnerable system is penetration testing. An organization may choose to do either or both as per their requirement. However, it's worth noting that a penetration test cannot be successful if a comprehensive vulnerability assessment hasn't been performed first.

A security assessment is nothing but detailed reviews of the security of a system, application, or other tested environments. During a security assessment, a trained professional conducts a risk assessment that uncovers potential vulnerabilities in the target environment that may allow a compromise and makes suggestions for mitigation, as required.

Like security testing, security assessments also normally include the use of testing tools but go beyond automated scanning and manual penetration tests. They also include a comprehensive review of the surrounding threat environment, present and future probable risks, and the asset value of the target environment.

The main output of a security assessment is generally a detailed assessment report intended for an organization's top management and contains the results of the assessment in nontechnical language. It usually concludes with precise recommendations and suggestions for improvising the security posture of the target environment.

A security audit often employs many of the similar techniques followed during security assessments but are required to be performed by independent auditors. An organization's internal security staff perform routine security testing and assessments. However, security audits differ from this approach. Security assessments and testing are internal to the organization and are intended to find potential security gaps.

Audits are similar to assessments but are conducted with the intent of demonstrating the effectiveness of security controls to a relevant third party. Audits ensure that there's no conflict of interest in testing the control effectiveness. Hence, audits tend to provide a completely unbiased view of the security posture.

The security assessment reports and the audit reports might look similar; however, they are both meant for different audiences. The audience for the audit report mainly includes higher management, the board of directors, government authorities, and any other relevant stakeholders.

There are two main types of audits:

• Internal audit: The organization's internal audit team performs the internal audit. The internal audit reports are intended for the organization's internal audience. It is ensured that the internal audit team has a completely independent reporting line to avoid conflicts of interest with the business processes they assess.
• External audit: An external audit is conducted by a trusted external auditing firm. External audits carry a higher degree of external validity since the external auditors virtually don't have any conflict of interest with the organization under assessment. There are many firms that perform external audits, but most people place the highest credibility with the so-called big four audit firms:
  • Ernst & Young
  • Deloitte & Touche
  • PricewaterhouseCoopers
  • KPMG

Audits performed by these firms are generally considered acceptable by most investors and governing bodies and regulators.