Penetration testing steps

Penetration testing essentially goes through multiple steps, based on the chosen methodology. In our case, we will study every phase according to the PTES.

Pre-engagement

Before conducting penetration testing, pre-engagement interactions between the pentester and the client should be established. This is a very important phase because you can view pentesting as an information technology project. Like any IT project, it needs great planning capabilities. Pentesting is not a set of technical steps but requires many management and organizational skills. An effective pentesting would start with a meeting with the client to have a crystal understanding of all their needs and vision. As a result of the meeting, a test plan will be developed. It will describe in detail how the pentest will be conducted. Many important items need to be taken care of during the pre-engagement phase.

The objectives and scope

It specifies the target of the pentesting, including the scope (IP addresses and hosts), in a very detailed way. In general, it also contains what assets are to be tested and what is out of bounds. The pre-engagement must also include the time period of the penetration testing mission.

A get out of jail free card

Hacking is an illegal act, so you need to assure that all your work will be done in a legal way. A get out of jail card usually signed by a high-level manager will be enough to get you out of trouble. Here the word "card" is a contract between the two parties that should solve the legal problem if any.

Emergency contact information

To avoid any panic situations when you find something serious, a predefined contact information list is a good idea to ensure a fast and efficient communication channel when it's needed. For example, If you are having an overwhelming network traffic issue because of an intensive automation tool, you need to contact the network engineer.

To avoid this inconvenience, it is better to discuss the availability of support in such situations with the stakeholders before conducting the penetration test.

Payment information

Payment information indicates the payment terms of the penetration testing. When discussing the schedule of the testing, the pentester should also discuss payment arrangements. During the negotiation, you can discuss the payment structure, for example, getting paid after delivering the final report, half amount up-front before conducting the pentesting, or according to a payment schedule. Also, non-payment penalties could be added to the agreement.

Non-disclosure agreement

The aim of a signed non-disclosure agreement (NDA) is to make the pentester commit to keeping all the confidential information and the findings safe. During penetration testing, you will be exposed to a certain amount of data with different classification ranks. That is why, it is a wise decision to sign a document to reassure the upper management that all the collected information is protected.

Intelligence gathering

The intelligence gathering stage is when the pentester searches for all available information about the organization from public sources. At the end of this phase, he will have a clear view of the network (domain name, IP ranges, TCP/UDP services, and authentication mechanisms), the systems (user/group names, system banners, and system architecture), and organizational information (employee details, press releases, and location). It depends on the type of pentesting (black, white, or gray). Implementing a good intelligence gathering methodology will facilitate the work in later steps.

The fuel of intelligence gathering is to get publicly available information from different sources. Intelligence gathering is not important in information security and penetration testing, but it is vital for national security, and as many concepts are inspired by the military strategies, in the cyber security field intelligence gathering is also inspired by the battlefields. But in a penetration testing context, all the techniques in this phase should be legal because good intentions do not mean breaking the law, that is why, we said publicly available information. If it is not, the case will be considered as industrial espionage. According to International Trade Commission estimates, current annual losses to US industries due to corporate espionage to be over $70 billion.

Intelligence gathering not only helps improve the security position of the organization, but it gives managers an eagle eye on the competition, and it results in better business decisions. Basically every intelligence gathering operation basically is done following a structured methodology.

Public intelligence

Public intelligence is the process of gathering all possible information about the target, using publicly available sources, and not only searching for it but also archiving it. The term is generally used by government agencies for national security operations. A penetration tester should also adopt such a state of mind and acquire the required skills to gather and classify information. In the era of huge amounts of data, the ability to extract useful information from it is a must.

Social engineering attacks

Social engineering attacks are when employees or others are psychologically deceived into providing sensitive information. Social engineering is the art of manipulating people to gain information about the users in order to ascertain sensitive information, such as login credentials or classified information. Using a human quality like trust in a deceptive way always shows that the human is the weakest layer in information security. One of the social engineering techniques is phishing, which is a technical method of social engineering. As we all know, Phishing is sending an email or text message that appears to come from a legitimate institution and tricking the user to type his login credentials. Spear-phishing is the same technique, but in a more specific range, such as sending phishing emails to short lists of high profile contacts

Physical analysis

Physical security is really critical in the information security landscape. Identifying physical equipment plays a huge role in intelligence gathering.

Information system and network analysis

This technique searches out information about the target including network services, devices, domains, and the information system information.

There are many intelligence gathering categories: human intelligence, signal intelligence, open source intelligence, imagery intelligence, and geospatial intelligence.

Human intelligence

Human intelligence (HUMINT) is the process of collecting information about human targets, with or without interaction with them, using many techniques such as taking photographs and video recording. There are three models of human intelligence:

Directed Gathering: This is a specific targeting operation. Usually, all the resources are meant to gather information about a unique target
Active Intelligence Gathering: This process is more specific and requires less investment, and it targets a specific environment.
Passive Intelligence Gathering: This is the foundation of human intelligence. The information is collected in opportunistic ways such as through walk-ins or referrals. So there is no specific target, except collecting information and trying to find something.

Signal intelligence

Signal intelligence (SIGINT) is the operation of gathering information by intercepting electronic signals and communications. It can be divided into two subcategories: communications intelligence (COMINT) and electronic intelligence (ELINT).

Open source intelligence

Open source intelligence (OSINT), as its name suggests, involves finding information about a defined target using available sources online. It can be done using many techniques:

Conducting search queries in many search engines
Gaining information from social media networks
Searching in deep web directories and the hidden wiki
Using forum and discussion boards

For example, if you want to search for a specific employee, you can use a theHarvester tool, and it will help find all public information about that person.

You can get theHarvester from its GitHub repository using this command from your console:

git clone https://github.com/laramies/theHarvester

Then, type ./theHarvester to run the script.

For example, if you want to collect information about a targetwebsite using Google search, simply run the following command:

theharvester -d   targetwebsite.org  -l 100 -b google

Here, the -l option is the limited number of results and -b indicates the search engine. In our case, we used the Google search engine:

Do you know that the known web represents only 4% of the internet. There is another space called the deep web. It contains 7,500 terabytes of information that means more than 500 billion pages.

It is an advantage to gather information from the hidden web, not only for reconnaissance purposes but for competitive intelligence. To access the deep web, you simply have to download the Tor Browser via its official website https://www.torproject.org/ and install it. Open the browser and hit Connect to access the network:

Now, you are surfing the hidden web. You can use the hidden wiki for Tor websites from this link, http://wiki5kauuihowqi5.onion (they are represented as DomainName.onion), or simply use the DuckDuckGo search engine:

Not only you can search for personal identifiable information, but you can also search for online devices and even industrial control systems. For example, you can check www.shodan.io. This search engine will help you find devices online. The following screenshot is publicly available information about wind turbines searched by Shodan.io:

To discover the great potential of the Shodan search engine, let's take a glimpse into the power of this giant. First, go to www.shodan.io and create a new account:

Use the search bar to enter a search query, or you can simply hit a predefined category: Netcams, default password, dreambox, industrial control systems, and so on. This is a snippet of the most popular search tags:

Let's hit Netcams as a demonstration. According to the screenshot listed as follows, the search engine found at least 8,632 publicly available sources of Netcam information, including their IP addresses with detailed descriptions about them:

Also, you can use a real-time map to search online devices such as routers:

Imagery intelligence

In the battlefield, imagery intelligence (IMINT) is the process of analyzing images and videos from different sources and devices such as electronic display images and infrared cameras. In penetration testing, image intelligence also works in the same way, where it is the operation of identifying information about the target using different photos and videos from different public resources as follows:

Social media (Facebook, LinkedIn, and so on) videos
Reverse searched photos for other editions
Live streams

There are many image analysis tools that you can use to extract data from an image. One of them is ExifTool. It is a small tool used to extract juicy information about a defined image. Like the following graph, just download ExifTool from this link, https://www.sno.phy.queensu.ca/~phil/exiftool/, and type ./exiftool image.png:

Geospatial intelligence

Geospatial intelligence (GEOINT) is the exploitation and analysis of imagery and geospatial information to describe, assess and visualize a defined area. The term GEOINT has become associated with information security and penetration testing. Identifying and collecting information about an organization will give penetration testers the ability to anticipate the physical intrusion of the organization. Thus, the role of a penetration tester is to make sure that data and sensitive information is safe from external threats.
There are many available sources to check for geospatial information. Google Maps is a free geospatial service provided by Google. The following is the result of a search, using a Google Map query:

Threat modeling

Threat modeling is a security approach to identify threats against the infrastructure of an organization. Modeling and quantifying are always wise decisions in information security, and especially in penetration testing. Measuring threats in a realistic way will help penetration testers make good decisions later. The aim of this structured approach is the identification and ranking of threats and assets, using a method that aligns with the business needs of the organization, and then mapping them.

In order to perform effective threat modeling, the penetration tester goes through five analysis steps.

Business asset analysis

During the business asset analysis, the pentester focuses on the assets by gathering any related documents about the assets, and in other situations, conducting interviews within the organization. It could include information about the following:

Infrastructure design
System configuration
User account credentials
Privileged user account credentials

Information about the technical assets is not sufficient to obtain efficient modeling. Penetration testers should gather information about all the policies and the procedures of the organization, and sometimes, the organization plan, if needed.

Business process analysis

Business is the central point in information security. A wise information security analysis will certainly assure that the organization works in a proper way and generates revenues. All the assets that are in a relationship with business processes are required to be mapped and analyzed, starting with the most critical ones. The following are the assets:

Information assets
Human assets
Third-party assets

Threat agents analysis

During this type of analysis, all the threats based on the location metric are mapped. We can divide threats into two categories—internal and external threats. Employees of the organizations, including the upper management, are also part of this classification because humans are the weakest layer when it comes to information security.

Threat capability analysis

After having a clear understanding about threat agents, now it is time to check for any available tools, exploits, and payloads currently out there that could be used against the organization infrastructure, in addition to analyzing the possible communication mechanisms.

Motivation modeling

A penetration tester could model the motivations behind an attack. In competitive environments and volatile businesses, motivation modeling should be added to the pentester checklist.

Vulnerability analysis

Threats are a serious problem for people and organizations. A clear understanding of vulnerability analysis is important to ensure that wise managerial decisions are taken and that a secure environment is built as a result of correctly identifying and mitigating such potential threats. Unfortunately, this is still a challenging area for information professionals because threats are becoming more sophisticated and hard to detect every day. Vulnerability assessment is the process of identifying, measuring, and classifying vulnerabilities in an information system. Vulnerability analysis is a critical skill for every pentester.

There is a big misunderstanding when it comes to vulnerability assessment. Many penetration testers confuse vulnerability analysis with penetration testing. In fact, penetration testing is simulating an attack, whereas vulnerability assessment is intended to identify vulnerabilities in a specific area. You can view it as a scanning operation.

A vulnerability management life cycle goes through the following six main phases:

Identification and discovery: During this phase, the pentester tries to identify all the assets within the discussed scope, including open services and operating systems and tries to detect common potential vulnerabilities in an information system, usually using automation tools and vulnerability scanners.
Prioritizing and classification: The penetration tester prioritizes the assets based on sensitivity criteria or based on categories. You can also prioritize vulnerabilities using a ranking system, for example, using the Common Vulnerability Scoring System (CVSS) for the Common Vulnerabilities and Exposures (CVE) vulnerabilities.
Assessment: This involves documenting analyzed risks. The pentester must make a decision about the risk acceptance after an evaluation process. When conducting a vulnerability assessment, you need to validate every found vulnerability. Using vulnerability scanners is important to detect potential vulnerabilities, but penetration testers need to verify every one of them to avoid false positive and incorrect flags.
Report: During this phase, the pentester shows the results of the conducted vulnerability assessment including the number of issues and trends, accompanied by graphical representations of the obtained artifacts.
Remediate: This is a detailed roadmap that includes recommendations and the steps required to remediate and fix vulnerabilities, not only technically, but it could include budgets, time slots, raking, and so on.
Verification: The final step involves verifying the fixed vulnerabilities after a follow-up check:

Vulnerability assessment with Nexpose

There are many vulnerability management tools currently available that can help the penetration tester during a vulnerability assessment mission, such us Beyond Security, Qualys, Core Security, and many other tools. One of the most well-known vulnerability management tools is the Rapid7's Nexpose. Nexpose assesses vulnerability in a defined infrastructure.

Installing Nexpose

You can install Nexpose by following the below steps:

Downloading the community edition from the official website,https://www.rapid7.com/products/nexpose/.
For the demonstration, we installed Nexpose for Windows 64 edition. You can use the Linux edition as well:

Fill in the required information and move to the next steps:

Starting Nexpose

To use Nexpose, you just need to navigate to http://localhost:3780 and enter your credentials.

Start a scan

To start a Nexpose scan, open a project, click on Create and select Site, for example. Then, enter a target IP or an IP range to start a scan:

After the scan is finished, you can generate a scanning report:

Exploitation

By this stage, the penetration tester already has what he needs to launch his attack. Now he only needs to bypass security controls to gain access to the infrastructure system. During this phase, the penetration tester wears a black hat and tries to gain access to the infrastructure from a malicious hacker's perspective. After a good threat analysis, now it is time to exploit every vulnerability. In order to exploit these vulnerabilities, you can use a variety of automation tools and manual testing. The most famous exploitation tool is Metasploit, which is a must in every penetration tester's arsenal.

To explore the exploits, a user can use the show exploits command.

Post-exploitation

Getting root privileges is not the end of the road. As discussed before, maintaining access is an essential phase in hacking methodologies, thus post-exploitation is required to not only maintain access but to spread into the infrastructure, to further compromise the system. This phase is critical; the penetration tester simulates an advanced attack; that is why, rules of engagement should be agreed before conducting post-exploitation. This shows and supports the importance of the first pentesting phase (pre-engagement) to protect your client and of course, protect yourself.

Based on the penetration testing execution standard, a post-exploitation phase should go through six sections.

Infrastructure analysis

Networks are the backbone of every modern organization and institution. So an infrastructure analysis will start by identifying the following:

Every network interface in the scope
Routing information
DNS servers and cashed DNS queries
Proxy servers
ARP entries

Not only network information but also identifying networking services is critical. They include the following:

Listening services
VPN connections
Mapping the neighbor devices using protocols such as Cisco Discovery Protocol and Link Layer Discovery Protocol

Pillaging

By definition, pillaging is gathering all possible information from the systems. Knowing where the data is located, for example, could help predict the pivoting techniques. To perform an effective penetration testing, you need to gather all, and not limited to, the following information, installed software and services:

Printers shares and security services
Database servers
Directory servers
Certificate authority services
Code management servers
Virtualization services

The exploitation of most of these services will be discussed in detail later in the following chapters.

High-profile targets

High profiles are extremely desirable from a hacker's perspective. Your job as a penetration tester is to make them top of your target list because compromising a high profile could result in compromising a business unit. Don't forget that curiosity and challenging spirit are motives for black hat hackers. That is why, C-level profiles are highly targeted.

Data exfiltration

During data exfiltration, the pentester maps all the exfiltration paths. The aim of this step is to make sure that there is no data leaving the organization in a sneaky way. Analyzing the data flow is of high priority, whereas data is the center of attention for hackers.

Persistence

Backdoors and meterpreters are very common techniques to assure the persistence, even after rebooting the system. Also, creating new accounts with complex passwords will gain you some presence time.

Further penetration into infrastructure

Curiosity is a double-edged weapon. It is part of who we are as humans. Persistent attacks are not enough for a hungry hacker. So, a penetration tester will look for further techniques to compromise more systems and networks in the infrastructure, to gain more access. Some of these techniques are:

Ping sweep
Internal DNS enumeration
Install uploaders
Services enumeration
Port forwarding
VPN to internal networks pivoting

Cleanup

Finally, the penetration tester must clean up the compromised systems of any used scripts, binaries, new accounts, and configurations during the previous post-exploitation steps.

Reporting

The final phase of pentesting is reporting. This is a deliverable document that includes all the findings and the processes conducted during the pentest mission. This step is very important for many reasons. The pentester needs to write a legible report so that every detail could be retested another time. Also, it should be comprehensible by the management board. Every report must be very clear and meaningful for both the technical and non-technical sides. To achieve a good pentesting report for different types of people, it should contain the following sections.

Executive summary

This section gives a high-level glimpse of the findings and specifies the main aims of the penetration testing. The target audience of this section is the upper management because they care about the security of the organization, more than the technical details. That is why, in an executive summary, it is not recommended you mention the technical specifications of the findings. The executive summary includes the following:

A background explains the purpose of the penetration testing and an explanation of some technical terms for the executive, if needed. The upper management, after reading the background, will have a clear idea about the goal and the expected results of the penetration testing.
An overall position relating to the effectiveness of the test by highlighting some security issues, such as according to the PTES standard, the business is lacking an effective patch management process.
Risk score is a general overview of risk ranking based on a predefined scoring system in the pre-engagement phase. Usually, we use the high/low scoring metrics or a numerical scale.
Recommendation summary specifies the required steps and methods to remediate the security issues discussed in the previous point.
Strategic roadmap indicates a detailed short- to long-term roadmap to enhance the security of an organization, based on ordered objectives.

Technical report

This section is made for technical managers and information technology staff. It includes detailed information about all the conducted steps and operations. It is structured the following way: