In this article by Kevin Cardwell, the author of the book Building Virtual Pentesting Labs for Advanced Penetration Testing, we will discuss the role that pen testing plays in the professional security testing framework. We will discuss the following topics:

• Defining security testing
• An abstract security testing methodology
• Myths and misconceptions about pen testing

(For more resources related to this topic, see here.)

If you have been doing penetration testing for some time and are very familiar with the methodology and concept of professional security testing, you can skip this article or just skim it. But you might learn something new or at least a different approach to penetration testing. We will establish some fundamental concepts in this article.

Security testing

If you ask 10 consultants to define what security testing is today, you will more than likely get a variety of responses. Here is the Wikipedia definition:

"Security testing is a process and methodology to determine that an information system protects and maintains functionality as intended."

In my opinion, this is the most important aspect of penetration testing. Security is a process and not a product. I'd also like to add that it is a methodology and not a product.

Another component to add to our discussion is the point that security testing takes into account the main areas of a security model. A sample of this is as follows:

• Authentication
• Authorization
• Confidentiality
• Integrity
• Availability
• Non-repudiation

Each one of these components has to be considered when an organization is in the process of securing their environment. Each one of these areas in itself has many subareas that also have to be considered when it comes to building a secure architecture. The lesson is that when testing security, we have to address each of these areas.

Authentication

It is important to note that almost all systems and/or networks today have some form of authentication and, as such, it is usually the first area we secure. This could be something as simple as users selecting a complex password or us adding additional factors to authentication, such as a token, biometrics, or certificates. No single factor of authentication is considered to be secure by itself in today's networks.

Authorization

Authorization is often overlooked since it is assumed and not a component of some security models. That is one approach to take, but it's preferred to include it in most testing models, as the concept of authorization is essential since it is how we assign the rights and permissions to access a resource, and we would want to ensure it is secure. Authorization enables us to have different types of users with separate privilege levels coexist within a system. We do this when we have the concept of discretionary access, where a user can have administrator privileges on a machine or assume the role of an administrator to gain additional rights or permissions, whereas we might want to provide limited resource access to a contractor.

Confidentiality

Confidentiality is the assurance that something we want to be protected on the machine or network is safe and not at risk of being compromised. This is made harder by the fact that the protocol (TCP/IP) running the Internet today is a protocol that was developed in the early 1970s. At that time, the Internet consisted of just a few computers, and now, even though the Internet has grown to the size it is today, we are still running the same protocol from those early days. This makes it more difficult to preserve confidentiality. It is important to note that when the developers created the protocol and the network was very small, there was an inherent sense of trust on who you could potentially be communicating with. This sense of trust is what we continue to fight from a security standpoint today. The concept from that early creation was and still is that you can trust data received to be from a reliable source. We know now that the Internet is at this huge size and that is definitely not the case.

Integrity

Integrity is similar to confidentiality, in that we are concerned with the compromising of information. Here, we are concerned with the accuracy of data and the fact that it is not modified in transit or from its original form. A common way of doing this is to use a hashing algorithm to validate that the file is unaltered.

Availability

One of the most difficult things to secure is availability, that is, the right to have a service when required. The irony of availability is that a particular resource is available to one user, and it is later available to all. Everything seems perfect from the perspective of an honest/legitimate user. However, not all users are honest/legitimate, and due to the sheer fact that resources are finite, they can be flooded or exhausted; hence, is it is more difficult to protect this area.

Non-repudiation

Non-repudiation makes the claim that a sender cannot deny sending something after the fact. This is the one I usually have the most trouble with, because a computer system can be compromised and we cannot guarantee that, within the software application, the keys we are using for the validation are actually the ones being used. Furthermore, the art of spoofing is not a new concept. With these facts in our minds, the claim that we can guarantee the origin of a transmission by a particular person from a particular computer is not entirely accurate.

Since we do not know the state of the machine with respect to its secureness, it would be very difficult to prove this concept in a court of law.

All it takes is one compromised machine, and then the theory that you can guarantee the sender goes out the window. We won't cover each of the components of security testing in detail here, because that is beyond the scope of what we are trying to achieve.

The point I want to get across in this article is that security testing is the concept of looking at each and every one of these and other components of security, addressing them by determining the amount of risk an organization has from them, and then mitigating that risk.

An abstract testing methodology

As mentioned previously, we concentrate on a process and apply that to our security components when we go about security testing. For this, I'll describe an abstract methodology here.

We will define our testing methodology as consisting of the following steps:

1. Planning
2. Nonintrusive target search
3. Intrusive target search
4. Data analysis
5. Reporting

Planning

Planning is a crucial step of professional testing. But, unfortunately, it is one of the steps that is rarely given the time that is essentially required. There are a number of reasons for this, but the most common one is the budget: clients do not want to provide consultants days and days to plan their testing. In fact, planning is usually given a very small portion of the time in the contract due to this reason. Another important point about planning is that a potential adversary is going to spend a lot of time on it. There are two things we should tell clients with respect to this step that as a professional tester we cannot do but an attacker could:

• 6 to 9 months of planning: The reality is that a hacker who targets someone is going to spend a lot of time before the actual attack. We cannot expect our clients to pay us for 6 to 9 months of work just to search around and read on the Internet.
• Break the law: We could break the law and go to jail, but it is not something that is appealing for most. Additionally, being a certified hacker and licensed penetration tester, you are bound to an oath of ethics, and you can be pretty sure that breaking the law while testing is a violation of this code of ethics.

Nonintrusive target search

There are many names that you will hear for nonintrusive target search. Some of these are open source intelligence, public information search, and cyber intelligence. Regardless of which name you use, they all come down to the same thing: using public resources to extract information about the target or company you are researching. There is a plethora of tools that are available for this. We will briefly discuss those tools to get an idea of the concept, and those who are not familiar with them can try them out on their own.

Nslookup

The nslookup tool can be found as a standard program in the majority of the operating systems we encounter. It is a method of querying DNS servers to determine information about a potential target. It is very simple to use and provides a great deal of information. Open a command prompt on your machine, and enter nslookup www.packtpub.com. This will result in output like the following screenshot: 

As you can see, the response to our command is the IP address of the DNS server for the www.packtpub.com domain. If we were testing this site, we would have explored this further. Alternatively, we may also use another great DNS-lookup tool called dig. For now, we will leave it alone and move to the next resource.

Central Ops

The https://centralops.net/co/ website has a number of tools that we can use to gather information about a potential target. There are tools for IP, domains, name servers, e-mail, and so on. The landing page for the site is shown in the next screenshot:

The first thing we will look at in the tool is the ability to extract information from a web server header page: click on TcpQuery, and in the window that opens, enter www.packtpub.com and click on Go. An example of the output from this is shown in the following screenshot:

As the screenshot shows, the web server banner has been modified and says packt. If we do the query against the www.packtpub.com domain we have determined that the site is using the Apache web server, and the version that is running; however we have much more work to do in order to gather enough information to target this site. The next thing we will look at is the capability to review the domain server information. This is accomplished by using the domain dossier. Return to the main page, and in the Domain Dossier dialog box, enter yahoo.com and click on go. An example of the output from this is shown in the following screenshot: 

There are many tools we could look at, but again, we just want to briefly acquaint ourselves with tools for each area of our security testing procedure. If you are using Windows and you open a command prompt window and enter tracert www.microsoft.com, you will observe that it fails, as indicated in this screenshot: 

The majority of you reading this article probably know why this is blocked; for those of you who do not, it is because Microsoft has blocked the ICMP protocol, which is what the tracert command uses by default. It is simple to get past this because the server is running services; we can use those protocols to reach it, and in this case, that protocol is TCP. If you go to http://www.websitepulse.com/help/testtools.tcptraceroute-test.html and enter www.microsoft.com in the IP address/domain field with the default location and conduct the TCP Traceroute test, you will see it will now be successful, as shown in the following screenshot:

As you can see, we now have additional information about the path to the potential target; moreover, we have additional machines to add to our target database as we conduct our test within the limits of the rules of engagement.

The Wayback Machine

The Wayback Machine is proof that nothing that has ever been on the Internet leaves! There have been many assessments in which a client informed the team that they were testing a web server that hadn't placed into production, and when they were shown the site had already been copied and stored, they were amazed that this actually does happen. I like to use the site to download some of my favorite presentations, tools, and so on, that have been removed from a site or, in some cases, whose site no longer exists. As an example, one of the tools used to show students the concept of steganography is the tool infostego. This tool was released by Antiy Labs, and it provided students an easy-to-use tool to understand the concepts. Well, if you go to their site at http://www.antiy.net/, you will find no mention of the tool—in fact, it will not be found on any of their pages. They now concentrate more on the antivirus market. A portion from their page is shown in the following screenshot:

Now, let's try and use the power of the Wayback Machine to find our software. Open the browser of your choice and go to www.archive.org. The Wayback Machine is hosted there and can be seen in the following screenshot:

As indicated, there are 491 billion pages archived at the time of writing this article. In the URL section, enter www.antiy.net and hit Enter. This will result in the site searching its archives for the entered URL. After a few moments, the results of the search will be displayed. An example of this is shown in the following screenshot:

We know we don't want to access a page that has been recently archived, so to be safe, click on 2008. This will result in the calendar being displayed and showing all the dates in 2008 on which the site was archived. You can select any one that you want; an example of the archived site from December 18 is shown in the following screenshot: as you can see, the infostego tool is available, and you can even download it! Feel free to download and experiment with the tool if you like.

Shodan

The Shodan site is one of the most powerful cloud scanners available. You are required to register with the site to be able to perform the more advanced types of queries. To access the site, go to https://www.shodan.io/. It is highly recommended that you register, since the power of the scanner and the information you can discover is quite impressive, especially after registration. The page that is presented once you log in is shown in the following screenshot:

The screenshot shows recently shared search queries as well as the most recent searches the logged-in user has conducted. This is another tool you should explore deeply if you do professional security testing. For now, we will look at one example and move on, since an entire article could be written just on this tool. If you are logged in as a registered user, you can enter iphone us into the search query window. This will return pages with iphone in the query and mostly in the United States, but as with any tool, there will be some hits on other sites as well.

An example of the results of this search is shown in the following screenshot:

Intrusive target search

This is the step that starts the true hacker-type activity. This is when you probe and explore the target network; consequently, ensure that you have with you explicit written permission to carry out this activity.

Never perform an intrusive target search without permission, as this written authorization is the only aspect which differentiates you and a malicious hacker. Without it, you are considered a criminal like them.

Within this step, there are a number of components that further define the methodology.

Find live systems

No matter how good our skills are, we need to find systems that we can attack. This is accomplished by probing the network and looking for a response. One of the most popular tools to do this with is the excellent open source tool nmap, written by Fyodor. You can download nmap from https://nmap.org/, or you can use any number of toolkit distributions for the tool. We will use the exceptional penetration-testing framework Kali Linux. You can download the distribution from https://www.kali.org/. Regardless of which version of nmap you explore with, they all have similar, if not the same, command syntax. In a terminal window, or a command prompt window if you are running it on Windows, type nmap –sP <insert network IP address>. The network we are scanning is the 192.168.4.0/24 network; yours more than likely will be different. An example of this ping sweep command is shown in the following screenshot: 

We now have live systems on the network that we can investigate further. For those of you who would like a GUI tool, you can use Zenmap.

Discover open ports

Now that we have live systems, we want to see what is open on these machines. A good analogy to a port is a door, and it's that if the door is open, I can approach it. There might be things that I have to do once I get to the door to gain access, but if it is open, then I know it is possible to get access, and if it is closed, then I know I cannot go through that door. Furthermore, we might need to know the type of lock that is on the door, because it might have weaknesses or additional protection that we need to know about. The same is with ports: if they are closed, then we cannot go into that machine using that port. We have a number of ways to check for open ports, and we will continue with the same theme and use nmap. We have machines that we have identified, so we do not have to scan the entire network as we did previously—we will only scan the machines that are up. Additionally, one of the machines found is our own machine; therefore, we will not scan ourselves—we could, but it's not the best plan. The targets that are live on our network are 1, 2, 16, and 18. We can scan these by entering nmap –sS 192.168.4.1,2,16,18. Those of you who want to learn more about the different types of scans can refer to http://nmap.org/book/man-port-scanning-techniques.html. Alternatively, you can use the nmap –h option to display a list of options. The first portion of the stealth scan (not completing the three-way handshake) result is shown in the following screenshot:

Discover services

We now have live systems and openings that are on the machine. The next step is to determine what, if anything, is running on the ports we have discovered—it is imperative that we identify what is running on the machine so that we can use it as we progress deeper into our methodology. We once again turn to nmap. In most command and terminal windows, there is history available; hopefully, this is the case for you and you can browse through it with the up and down arrow keys on your keyboard. For our network, we will enter nmap –sV 192.168.4.1. From our previous scan, we've determined that the other machines have all scanned ports closed, so to save time, we won't scan them again. An example of this is shown in the following screenshot:

From the results, you can now see that we have additional information about the ports that are open on the target. We could use this information to search the Internet using some of the tools we covered earlier, or we could let a tool do it for us.

Enumeration

Enumeration is the process of extracting more information about the potential target to include the OS, usernames, machine names, and other details that we can discover. The latest release of nmap has a scripting engine that will attempt to discover a number of details and in fact enumerate the system to some aspect. To process the enumeration with nmap, use the -A option. Enter nmap -A 192.168.4.1. Remember that you will have to enter your respective target address, which might be different from the one mentioned here. Also, this scan will take some time to complete and will generate a lot of traffic on the network. If you want an update, you can receive one at any time by pressing the spacebar. This command's output is quite extensive; so a truncated version is shown in the following screenshot:

As you can see, you have a great deal of information about the target, and you are quite ready to start the next phase of testing. Additionally, we have the OS correctly identified; until this step, we did not have that.

Identify vulnerabilities

After we have processed the steps up to this point, we have information about the services and versions of the software that are running on the machine. We could take each version and search the Internet for vulnerabilities, or we could use a tool—for our purposes, we will choose the latter. There are numerous vulnerability scanners out there in the market, and the one you select is largely a matter of personal preference. The commercial tools for the most part have a lot more information and details than the free and open source ones, so you will have to experiment and see which one you prefer. We will be using the Nexpose vulnerability scanner from Rapid7. There is a community version of their tool that will scan a limited number of targets, but it is worth looking into. You can download Nexpose from http://www.rapid7.com/. Once you have downloaded it, you will have to register, and you'll receive a key by e-mail to activate it. I will leave out the details of this and let you experience them on your own. Nexpose has a web interface, so once you have installed and started the tool, you have to access it. You can access it by entering https://localhost:3780. It seems to take an extraordinary amount of time to initialize, but eventually, it will present you with a login page, as shown in the following screenshot:

The credentials required for login will have been created during the installation. It is quite an involved process to set up a scan, and since we are just detailing the process and there is an excellent quick start guide available, we will just move on to the results of the scan. We will have plenty of time to explore this area as the article progresses. The result of a typical scan is shown in the following screenshot:

As you can see, the target machine is in bad shape. One nice thing about Nexpose is the fact that since they also own Metasploit, they will list the vulnerabilities that have a known exploit within Metasploit.

Exploitation

This is the step of the security testing that gets all the press, and it is, in simple terms, the process of validating a discovered vulnerability. It is important to note that it is not a 100-percent successful process—some vulnerabilities will not have exploits and some will have exploits for a certain patch level of the OS but not others. As I like to say, it is not an exact science and in reality is an infinitesimal part of professional security testing, but it is fun, so we will briefly look at the process. We also like to say in security testing that we have to validate and verify everything a tool reports to our client, and that is what we try to do with exploitation. The point is that you are executing a piece of code on a client's machine, and this code could cause damage. The most popular free tool for exploitation is the Rapid7-owned tool Metasploit. There are entire articles written on using the tool, so we will just look at the results of running it and exploiting a machine here. As a reminder, you have to have written permission to do this on any network other than your own; if in doubt, do not attempt it. Let's look at the options:

There is quite a bit of information in the options. The one we will cover is the fact that we are using the exploit for the MS08-067 vulnerability, which is a vulnerability in the server service. It is one of the better ones to use as it almost always works and you can exploit it over and over again. If you want to know more about this vulnerability, you can check it out here: http://technet.microsoft.com/en-us/security/bulletin/ms08-067. Since the options are set, we are ready to attempt the exploit, and as indicated in the following screenshot, we are successful and have gained a shell on the target machine. The process for this we will cover as we progress through the article. For now, we will stop here.

Here onward, it is only your imagination that can limit you. The shell you have opened is running at system privileges; therefore, it is the same as running a Command Prompt on any Windows machine with administrator rights, so whatever you can do in that shell, you can also do in this one. You can also do a number of other things, which you will learn as we progress through the article. Furthermore, with system access, we can plant code as malware: a backdoor or really anything we want. While we might not do that as a professional tester, a malicious hacker could do it, and this would require additional analysis to discover on the client's end.

Data analysis

Data analysis is often overlooked, and it can be a time-consuming process. This is the process that takes the most time to develop. Most testers can run tools and perform manual testing and exploitation, but the real challenge is taking all of the results and analyzing them. We will look at one example of this in the next screenshot. Take a moment and review the protocol analysis captured with the tool Wireshark—as an analyst, you need to know what the protocol analyzer is showing you. Do you know what exactly is happening? Do not worry, I will tell you after we have a look at the following screenshot:

You can observe that the machine with the IP address 192.168.3.10 is replying with an ICMP packet that is type 3 code 13; in other words, the reason the packet is being rejected is because the communication is administratively filtered. Furthermore, this tells us that there is a router in place and it has an access control list (ACL) that is blocking the packet. Moreover, it tells us that the administrator is not following best practices— absorbing packets and not replying with any error messages that can assist an attacker. This is just a small example of the data analysis step; there are many things you will encounter and many more that you will have to analyze to determine what is taking place in the tested environment. Remember: the smarter the administrator, the more challenging pen testing can become—which is actually a good thing for security!

Reporting

Reporting is another one of the areas in testing that is often overlooked in training classes. This is unfortunate since it is one of the most important things you need to master. You have to be able to present a report of your findings to the client. These findings will assist them in improving their security practices, and if they like the report, it is what they will most often share with partners and other colleagues. This is your advertisement for what separates you from others. It is a showcase that not only do you know how to follow a systematic process and methodology of professional testing, you also know how to put it into an output form that can serve as a reference going forward for the clients. At the end of the day, as professional security testers, we want to help our clients improve their security scenario, and that is where reporting comes in. There are many references for reports, so the only thing we will cover here is the handling of findings. There are two components we use when it comes to findings, the first of which is a summary-of-findings table. This is so the client can reference the findings early on in the report. The second is the detailed findings section. This is where we put all of the information about the findings. We rate them according to severity and include the following:

Description

This is where we provide the description of the vulnerability, specifically, what it is and what is affected.

Analysis and exposure

For this article, you want to show the client that you have done your research and aren't just repeating what the scanning tool told you. It is very important that you research a number of resources and write a good analysis of what the vulnerability is, along with an explanation of the exposure it poses to the client site.

Recommendations

We want to provide the client a reference to the patches and measures to apply in order to mitigate the risk of discovered vulnerabilities. We never tell the client not to use the service and/or protocol! We do not know what their policy is, and it might be something they have to have in order to support their business. In these situations, it is our job as consultants to recommend and help the client determine the best way to either mitigate the risk or remove it. When a patch is not available, we should provide a reference to potential workarounds until one is available.

References

If there are references such as a Microsoft bulletin number or a Common Vulnerabilities and Exposures (CVE) number, this is where we would place them.

Myths and misconceptions about pen testing

After more than 20 years of performing professional security testing, it is amazing to me really how many are confused as to what a penetration test is. I have on many occasions gone to a meeting where the client is convinced they want a penetration test, and when I explain exactly what it is, they look at me in shock. So, what exactly is a penetration test? Remember our abstract methodology had a step for intrusive target searching and part of that step was another methodology for scanning? Well, the last item in the scanning methodology, exploitation, is the step that is indicative of a penetration test. That's right! That one step is the validation of vulnerabilities, and this is what defines penetration testing. Again, it is not what most clients think when they bring a team in. The majority of them in reality want a vulnerability assessment. When you start explaining to them that you are going to run exploit code and all these really cool things on their systems and/or networks, they usually are quite surprised. The majority of the times, the client will want you to stop at the validation step. On some occasions, they will ask you to prove what you have found, and then you might get to show validation. I once was in a meeting with the IT department of a foreign country's stock market, and when I explained what we were about to do for validating vulnerabilities, the IT director's reaction was, "Those are my stock broker records, and if we lose them, we lose a lot of money!" Hence, we did not perform the validation step in that test.

Summary

In this article, we defined security testing as it relates to this article, and we identified an abstract methodology that consists of the following steps: planning, nonintrusive target search, intrusive target search, data analysis, and reporting. More importantly, we expanded the abstract model when it came to intrusive target searching, and we defined within that a methodology for scanning. This consisted of identifying live systems, looking at open ports, discovering services, enumeration, identifying vulnerabilities, and finally, exploitation.

Furthermore, we discussed what a penetration test is and that it is a validation of vulnerabilities and is associated with one step in our scanning methodology. Unfortunately, most clients do not understand that when you validate vulnerabilities, it requires you to run code that could potentially damage a machine or, even worse, damage their data. Because of this, once they discover this, most clients ask that it not be part of the test. We created a baseline for what penetration testing is in this article, and we will use this definition throughout the remainder of this article.

In the next article, we will discuss the process of choosing your virtual environment. 

Resources for Article: 

Further resources on this subject:

• CISSP: Vulnerability and Penetration Testing for Access Control [article]
• Web app penetration testing in Kali [article]
• BackTrack 4: Security with Penetration Testing Methodology [article]