Common pitfalls of vulnerability assessments, penetration testing, and red team exercises
In this section, we will discuss some misconceptions and limitations regarding traditional/classical vulnerability scanning, penetration testing, and red team exercises. Let us now understand the actual meaning of these three topics in simple terms and their limitations:
- Vulnerability Assessment (VA): The process of identifying vulnerabilities or security loopholes in a system or network through a vulnerability scanner. One of the misconceptions about VA is that it will let you find all of the known vulnerabilities; well, that’s not true. Limitations with VA include that only potential vulnerabilities are found, and it depends purely on the type of scanner that you utilize. It might also include a number of false positives and, to the business owner, there is no clear indication as to which ones do not pose a relevant risk and which one will be initially utilized by the attackers to gain access. The biggest pitfall of VA is false negatives, meaning the scanner did not find an issue that the system or application has.
- Penetration testing (pentesting): The process of safely simulating the hacking scenarios by exploiting vulnerabilities without much impact on the existing network or business. There is also a lower number of false positives since testers will try to validate the vulnerabilities and also attempt to exploit them. A limitation with pentesting is that it uses only currently known, publicly available exploits; mostly, these are a focus for project testing. We often hear from pentesters during an assessment, Yay! Got Root—but we never hear the question, what can you do with it? This could be due to various reasons such as project limitations, including the reporting of high-risk issues immediately to the client, or the client only being interested in one segment of the network and only wanting that part tested.
One of the misconceptions about the pentest is that it provides the attacker with a full view of the network, and you are safe once penetration testing has been performed. This is not the case when attackers have found a vulnerability in the business process of your secure application.
- Red Team Exercise (RTE): A focused process of evaluating the effectiveness of an organization to defend against cyber threats and improve its security by any possible means; during an RTE, we can discover multiple ways of achieving project objectives/scenarios and goals, such as complete coverage of activities with the defined project goal, including phishing (enticing a victim to enter sensitive information or download malicious content through emails), vishing (enticing a victim to provide or do some actions with malicious intent through phone calls), “WhatsApping” (engaging a victim through WhatsApp messenger with malicious intent), wireless, disk drops (USB and SSD), and physical penetration testing. The limitations with RTEs are time-bound, pre-defined scenarios and an assumed rather than real environment. Often, the RTE is run with a fully monitored mode for every technique, and tactics are executed according to the procedure, but this isn’t the case when a real attacker wants to achieve an objective.
Figure 1.1 showcases the difference between all three activities in terms of the length and breadth of their focus:
Figure 1.1: The three methods of assessing the vulnerability of systems and the breadth and depth to which they are successful
Often, all three different testing methodologies refer to the term hack or compromise. We will hack your network and show you where your weaknesses are; but wait, does the client or business owner understand the difference between these terms? How do we measure it? What are the criteria? And when do we know that the hack or compromise is complete? All the questions point to only one thing: what the purpose of the testing is, and what the primary goal in mind is.
