There is a general path that cybersecurity incidents follow during their lifetime. If the organization has a mature IR capability, it will have taken measures to ensure it is prepared to address an incident at each stage of the process. Each incident starts with the first time the organization becomes aware of an event or series of events indicative of malicious activity. This detection can come in the form of a security control alert or an external party informing the organization of a potential security issue. Once alerted, the organization moves through analyzing the incident through containment measures to bring the information system back to normal operations. There is no set IR process. One standard that is widely used is the National Institute of Standards and Technology (NIST) IR process. The following diagram, taken from the NIST Special Publication (SP) 800-61 shows how the NIST process flows in a cycle, with preparation as the starting point. A closer examination reveals that every incident is used to better prepare the organization for future incidents as the post-incident activity, and is utilized in preparation for the next incident:

Figure 1.1 – NIST IR process

The IR process can be broken down into the following six distinct phases, each with a set of actions the organization can take to address the incident:

• Preparation: Without good preparation, any subsequent IR is going to be disorganized and has the potential to make the incident worse. One of the critical components of preparation is the creation of an IR plan. Once a plan is in place with the necessary staffing, ensure that personnel detailed with IR duties are properly trained. This includes processes, procedures, and any additional tools necessary for the investigation of an incident. In addition to a plan, tools such as forensics hardware and software should be acquired and incorporated into the overall process. Finally, regular exercises should be conducted to ensure that the organization is trained and familiar with the process.
• Detection: The detection of potential incidents is a complex endeavor. Depending on the size of the organization, they may have over 100 million separate events per day. These events can be records of legitimate actions taken during the normal course of business or be indicators of potentially malicious activity. Couple this mountain of event data with other security controls constantly alerting to activity and you have a situation where analysts are inundated with data and must subsequently sift out the valuable pieces of signal from the vastness of network noise. Even today’s cutting-edge Security Information and Event Management (SIEM) tools lose their effectiveness if they are not properly maintained with regular updates of rulesets that identify which events qualify as a potential incident. The detection phase is that part of the IR process where the organization first becomes aware of a set of events that possibly indicates malicious activity. These events that have been detected and are indicative of malicious behavior are then classified as an incident. For example, a security analyst may receive an alert that a specific administrator account was in use when the administrator was on vacation. Detection may also come from external sources. An internet service provider (ISP) or law enforcement agency may detect malicious activity originating in an organization’s network and contact them to advise them of the situation. In other instances, users may be the first to indicate a potential security incident. This may be as simple as an employee contacting the help desk and informing a help desk technician that they received an Excel spreadsheet from an unknown source and opened it. They are now complaining that their files on the local system are being encrypted. In each case, an organization would have to escalate each of these events to the level of an incident (which we will cover a little later in this chapter) and begin the reactive process to investigate and remediate.
• Analysis: Once an incident has been detected, personnel from the organization or a trusted third party will begin the analysis phase. In this phase, personnel begins the task of collecting evidence from systems such as running memory, log files, network connections, and running software processes. Depending on the type of incident, this collection can take from as little as a few hours to several days. Once evidence is collected, it then needs to be examined. There are a variety of tools to conduct this analysis, many of which are explored in this book. With these tools, analysts attempt to ascertain what happened, what it affected, whether any other systems were involved, and whether any confidential data was removed. The ultimate goal of the analysis is to determine the root cause of the incident and reconstruct the actions of the threat actor, from initial compromise to detection.
• Containment: Once there is a solid understanding of what the incident is and which systems are involved, organizations can then move into the containment phase. In this phase, organizations take measures to limit the ability of threat actors to continue compromising other network resources, communicating with command and control (C2) infrastructures, or exfiltrating confidential data. Containment strategies can range from locking down ports and Internet Protocol (IP) addresses on a firewall to simply removing the network cable from the back of an infected machine. Each type of incident involves its own containment strategy, but having several options allows personnel to stop the bleeding at the source if they are able to detect a security incident before or while threat actors are pilfering data.
• Eradication and recovery: During the eradication phase, the organization removes the threat actor from the impacted network. In the case of a malware infection, the organization may run an enhanced anti-malware solution. Other times, infected machines must be wiped and reimaged. Other activities include removing or changing compromised user accounts. If an organization has identified a vulnerability that was exploited, vendor patches are applied or software updates are made. Recovery activities are very closely aligned with those that may be found in an organization’s business continuity (BC) or disaster recovery (DR) plans. In this phase of the process, organizations reinstall fresh operating systems or applications. They will also restore data on local systems from backups. As a due diligence step, organizations will also audit their existing user and administrator accounts to ensure that no accounts have been enabled by threat actors. Finally, a comprehensive vulnerability scan is conducted so that the organization is confident that any exploitable vulnerabilities have been removed.
• Post-incident activity: At the conclusion of the incident process is a complete review of the incident with all principal stakeholders. Post-incident activity includes a complete review of all actions taken during the incident. What worked and—more importantly—what did not work are important topics for discussion. These reviews are important because they may highlight specific tasks and actions that had either a positive or negative impact on the outcome of the IR. It is during this phase of the process that a written report is completed. Documenting the actions taken during the incident is critical to capture both what occurred and whether the incident will ever see the inside of a courtroom. For documentation to be effective, it should be detailed and show a clear chain of events with a focus on the root cause, if it was determined. Personnel involved in the preparation of this report should realize that stakeholders outside of IT might read this report. As a result, technical jargon or concepts should be explained.

Finally, the organizational personnel should update their own IR processes with any new information developed during the post-incident debrief and reporting. This incorporation of lessons learned is important as it makes future responses to incidents more effective.

The role of digital forensics

There is a misconception that is often held by people unfamiliar with the realm of IR, which is that IR is merely a digital forensics issue. As a result, they will often conflate the two terms. While digital forensics is a critical component of IR (and for this reason, we have included a number of chapters in this book that address digital forensics), there is more to addressing an incident than examining hard drives. It is best to think of forensics as a supporting function of the overall IR process. Digital forensics serves as the mechanism for understanding the technical aspects of an incident, potentially identifying the root cause, and discovering unidentified access or other malicious activity. For example, some incidents such as denial-of-service (DoS) attacks will require little to no forensic work. On the other hand, a network intrusion involving the compromise of an internal server and C2 traffic leaving the network will require extensive examination of logs, traffic analysis, and examination of memory. From this analysis, the root cause may be derived. In both cases, the impacted organization would be able to connect with the incident, but forensics plays a much more important role in the latter case.

IR is an information security function that uses the methodologies, tools, and techniques of digital forensics but goes beyond what digital forensics provides by addressing additional elements. These elements include containing possible malware or other exploits, identifying and remediating vulnerabilities, and managing various technical and non-technical personnel. Some incidents may require the analysis of host-based evidence or memory while others may only require a firewall log review but, in each, the responders will follow the IR process.

Responding to a data breach, ransomware attack, or another security incident should never be an ad hoc process. Undefined processes or procedures will leave an organization unable to both identify the extent of the incident and be able to stop the bleeding in sufficient time to limit the damage. Further, attempting to craft plans during an incident may, in fact, destroy critical evidence or—worse—create more problems.

Having a solid understanding of the IR process is just the first step to building this capability within an organization. What organizations need is a framework that puts processes to work utilizing the organization’s available resources. An IR framework describes the components of a functional IR capability within an organization. This framework is made up of elements such as personnel, policies, procedures, and implementation. It is through these elements that an organization builds its capability to respond to incidents.

The IR charter

The first step to building this capability is the decision by senior leadership that the risk to the organization is too significant not to address the possibility of a potential security incident. Once that point is reached, a senior member of the organization will serve as a project sponsor and craft an IR charter. This charter outlines key elements that will drive the creation of a computer security IR team (CSIRT).

Information

While there are several titles for IR teams, the term CERT, short for computer emergency response team, is often associated with the US-CERT through the United States Department of Homeland Security (US DHS) or the CERT Coordination Center (CERT/CC), through the Carnegie Mellon Software Engineering Institute (SEI). For our purposes, we will use the more generic CSIRT.

The IR charter should be a written document that addresses the following:

• Obtaining senior leadership support: In order to be a viable part of the organization, the CSIRT requires the support of the senior leadership within the organization. In a private-sector institution, it may be difficult to obtain the necessary support and funding, as the CSIRT itself does not provide value in the same way marketing or sales does. What should be understood is that the CSIRT acts as an insurance policy in the event the worst happens. In this manner, a CSIRT can justify its existence by reducing the impact of incidents and thereby reducing costs associated with a security breach or other malicious activity.
• Defining a constituency: A constituency clearly defines which organizational elements and domains the CSIRT has responsibility for. Some organizations have several divisions or subsidiaries that, for whatever reason, may not be part of the CSIRT’s responsibility. The constituency can be defined either as a domain such as local.example.com or an organization name such as ACME Inc. and associated subsidiary organizations.
• Creating a mission statement: Mission creep or the gradual expansion of the CSIRT’s responsibilities can occur without a clear definition of what the defined purpose of the CSIRT is. In order to counter this, a clearly defined mission statement should be included with the written information security plan. For example, the mission of the ACME Inc. CSIRT is to provide timely analysis and actions for security incidents that impact the confidentiality, integrity, and availability of ACME Inc. information systems and personnel.
• Determining service delivery: Along with a mission statement, a clearly defined list of services can also counter the risk of mission creep of the CSIRT. Services are usually divided into two separate categories—proactive and reactive services, as outlined here:
  • Proactive services: These include providing training for non-CSIRT staff, providing summaries on emerging security threats, testing and deployment of security tools such as endpoint detection and response (EDR) tools, and assisting security operations by crafting intrusion detection systems/intrusion prevention systems (IDS/IPS) alerting rules.
  • Reactive services: These primarily revolve around responding to incidents as they occur. For the most part, reactive services address the entire IR process. This includes the acquisition and examination of evidence, assisting in containment, eradication, and recovery efforts, and—finally—documenting the incident.

Another critical benefit of an expressly stated charter is to socialize the CSIRT with the entire organization. This is done to remove any rumors or innuendo about the purpose of the team. Employees of the organization may hear terms such as digital investigations or IR team and believe the organization is preparing secret police specifically designed to ferret out employee misconduct. To counter this, a short statement that includes the mission statement of the CSIRT can be made available to all employees. The CSIRT can also provide periodic updates to senior leadership on incidents handled to demonstrate the purpose of the team.

CSIRT team

Once the IR charter is completed, the next stage is to start staffing the CSIRT. Larger organizations with sufficient resources may be able to task personnel with IR duties full-time. Often, though, organizations will have to utilize personnel who have other duties outside IR. Personnel who comprise the internal CSIRT can be divided into three categories: core team, technical support, and organizational support. Everyone within the CSIRT fulfills a specific task. Building this capability into an organization takes more than just assigning personnel and creating a policy-and-procedure document. As with any major project initiative, there is a good deal of effort required in creating a functional CSIRT.

For each of the CSIRT categories, there are specific roles and responsibilities. This wide range of personnel is designed to provide guidance and support through a wide range of incidents, ranging from minor to catastrophic.

CSIRT core team

The CSIRT core team consists of personnel who have IR duties as their full-time job or assume IR activities when needed. In many instances, the core team is often made up of personnel assigned to the information security team. Other organizations can leverage personnel with expertise in IR activities. Here are some of the roles that can be incorporated into the core team:

• IR coordinator: This is a critical component of any CSIRT. Without clear leadership, the response to a potential incident may be disorganized or, with multiple individuals vying for control during an incident, a chaotic situation that can make the incident worse. In many instances, the IR coordinator is often the chief security officer (CSO), the chief information security officer (CISO), or the information security officer (ISO) as that individual often has overall responsibility for the security of the organization’s information. Other organizations may name a single individual who serves as the IR coordinator. The IR coordinator is responsible for the management of the CSIRT prior to, during, and after an incident. In terms of preparation, the IR coordinator will ensure that any plans or policies concerning the CSIRT are reviewed periodically and updated as needed. In addition, the IR coordinator is responsible for ensuring that the CSIRT team is appropriately trained and also oversees testing and training for CSIRT personnel.

During an incident, the IR coordinator is responsible for ensuring the proper response and remediation of an incident and guides the team through the entire IR process. One of the most important of these tasks during an incident is the coordination of the CSIRT with senior leadership. With the stakes of a data breach being high, senior leadership such as the chief executive officer (CEO) will want to be kept up-to-date in terms of critical information concerning an incident. It is the responsibility of the IR coordinator to ensure that senior leadership is fully informed of the activities associated with an incident, using clear and concise language. One stumbling block is that senior leaders within an organization may not have the acumen to understand the technical aspects of an incident, so it is important to speak in a language they will understand.

Finally, at the conclusion of an incident, the IR coordinator is responsible for ensuring that the incident is properly documented and that reports of the CSIRT activity are delivered to the appropriate internal and external stakeholders. In addition, a full debrief of all CSIRT activities is conducted, and lessons learned are incorporated into the CSIRT plan.

• CSIRT senior analyst(s): CSIRT senior analysts are personnel with extensive training and experience in IR and associated skills such as digital forensics or network data examination. They often have several years of experience conducting IR activities as either a consultant or as part of an enterprise CSIRT.

During the preparation phase of the IR process, they are involved in ensuring that they have the necessary skills and training to address their specific role in the CSIRT. They are also often directed to assist in the IR plan review and modification. Finally, senior analysts will often take part in training junior members of the team.

Once an incident has been identified, senior analysts will engage with other CSIRT members to acquire and analyze evidence, direct containment activities, and assist other personnel with remediation.

At the conclusion of an incident, senior analysts will ensure that both they and other personnel appropriately document the incident. This will include the preparation of reports to internal and external stakeholders. They will also ensure that any evidence is appropriately archived or destroyed, depending on the IR plan.

• CSIRT analyst(s): CSIRT analysts are personnel with CSIRT responsibilities that have less exposure or experience in IR activities. Oftentimes, they have only 1 or 2 years of experience in responding to incidents. As a result, they can perform a variety of activities, with some of those under the direction of senior analysts.

In terms of preparation-phase activities, analysts will develop their skills via training and exercises. They may also take part in reviews and updates to the IR plan. During an incident, they will be tasked with gathering evidence from potentially compromised hosts, network devices, or various log sources. Analysts will also take part in the analysis of evidence and assist other team members in remediation activities.

• Security operations center (SOC) analyst: Larger enterprises may have an in-house or contracted 24/7 SOC monitoring capability. Analysts assigned to the SOC will often serve as the point person when it comes to incident detection and alerting. As a result, having a SOC analyst as part of the team allows them to be trained in incident identification and response techniques and serve as an almost immediate response to a potential security incident.
• IT security engineer/analyst(s): Depending on the size of the organization, there may be personnel specifically tasked with the deployment, maintenance, and monitoring of security-related software such as anti-virus or hardware such as firewalls or SIEM systems. Having direct access to these devices is critical when an incident has been identified. The personnel assigned these duties will often have a direct role in the entire IR process.

The IT security engineer or analyst will be responsible for the preparation component of the IR process. They will be the primary resource to ensure that security applications and devices are properly configured to alert to possible incidents and to ensure that the devices properly log events so that a reconstruction of events can take place.

During an incident, they will be tasked with monitoring security systems for other indicators of malicious behavior. They will also assist other CSIRT personnel with obtaining evidence from security devices. Finally, after an incident, this personnel will be tasked with configuring security devices to monitor suspected behavior, to ensure that remediation activities have eradicated malicious activity on impacted systems.

Technical support personnel

Technical support personnel are those individuals within the organization who do not have CSIRT activities as part of their day-to-day operations, but rather have expertise or access to systems and processes that may be affected by an incident. For example, the CSIRT may need to engage a server administrator to assist the core team with acquiring evidence from servers such as memory captures, acquiring virtual systems, or offloading log files. Once completed, the server administrator’s role is completed and they may have no further involvement in the incident. Here are some of the personnel that can be of assistance to the CSIRT during an incident:

• Network architect/administrator: Often, incidents involve the network infrastructure. This includes attacks on routers, switches, and other network hardware and software. The network architect or administrator is vital for insight into what is normal and abnormal behavior for these devices, as well as for identifying anomalous network traffic. In incidents where the network infrastructure is involved, these support personnel can assist with obtaining network evidence such as access logs or packet captures.
• Server administrator: Threat actors often target systems within the network where critical or sensitive data is stored. These high-value targets often include domain controllers, file servers, or database servers. Server administrators can aid in acquiring log files from these systems. If the server administrator(s) are also responsible for the maintenance of the Active Directory structure, they may be able to assist with identifying new user accounts or changes to existing user or administrator accounts.
• Application support: Web applications are a prime target for threat actors. Flaws in coding that allow attacks such as Structured Query Language (SQL) injection or security misconfigurations are responsible for some security breaches. As a result, having application support personnel as part of the CSIRT facilitates the finding of information directly related to application attacks. These individuals will often be able to identify code changes or confirm vulnerabilities discovered during an investigation into a potential attack against an application.
• Desktop support: Desktop support personnel are often involved in maintaining controls such as data loss prevention and anti-virus on desktop systems. In the event of an incident, they can assist in providing the CSIRT with log files and other evidence. They may also be responsible for cleaning up infected systems during the remediation phase of an incident.
• Help desk: Depending on the organization, help desk personnel are the proverbial canary in the coal mine when it comes to identifying an incident. They are often the first individuals contacted when a user experiences the first signs of a malware infection or other malicious activity. Thus, help desk personnel should be involved in the training of CSIRT responses and their role in incident identification and escalation procedures. They may also assist with identifying additional affected personnel in the event of a widespread incident.

Organizational support personnel

Outside of the technical realm, other organizational members should also be included within the CSIRT. Organizational personnel can assist with a variety of non-technical issues that fall outside those that are addressed by the CSIRT core and technical support personnel. These include navigating the internal and external legal environment, assisting with customer communications, or supporting CSIRT personnel while on-site.

Here are some of the organizational support personnel that should be included in a CSIRT plan:

• Legal: Data breaches and other incidents carry a variety of legal issues along with them. Many countries now have breach notification laws where organizations are required to notify customers that their information was put at risk. Other compliance requirements such as the Health Insurance and Portability Act (HIPAA) and the Payment Card Industry Data Security Standard (PCI DSS) require the impacted organization to contact various external bodies and notify them of a suspected breach. Including legal representation early in the IR process will ensure that these notifications and any other legal requirements are addressed in a timely fashion. In the event that a breach has been caused by an internal source such as an employee or contractor, the impacted organization may want to recoup losses through civil action. Including legal representation early in the process will allow a more informed decision as to which legal process should be followed.
• Human resources (HR): A good deal of incidents that occur in organizations are perpetrated by employees or contractors. The investigation of actions such as fraud, all the way to massive data theft, may have to be investigated by the CSIRT. In the event that the target of the investigation is an employee or contractor, the HR department can assist with ensuring that the CSIRT’s actions are in compliance with applicable labor laws and company policies. If an employee or contractor is to be terminated, the CSIRT can coordinate with HR personnel so that all proper documentation concerning the incident is complete, to reduce the potential of a wrongful termination suit.
• Marketing/communications: If external clients or customers may be adversely impacted by an incident such as a DoS attack or data breach, the marketing or communications department can assist in crafting the appropriate message to assuage fears and ensure that those external entities are receiving the best information possible. When looking back at past data breaches where organizations attempted to keep the details to themselves and customers were not informed, there was a backlash against those organizations. Having a solid communications plan that is put into action early will go a long way in soothing any potentially adverse reactions from customers or clients.
• Facilities: The CSIRT may need access to areas after hours or for a prolonged time. The facilities department can assist the CSIRT in obtaining the necessary access in a timely manner. Facilities also may have access to additional meeting spaces for the CSIRT to utilize in the event of a prolonged incident that requires a dedicated workspace and infrastructure.
• Corporate security: The CSIRT may be called in to deal with the theft of network resources or other technology from the organization. Laptop and digital media theft are very common. Corporate security will often have access to surveillance footage from entrances and exits. They may also maintain access badges and visitor logs for the CSIRT to track the movement of employees and other personnel within the facility. This can allow a reconstruction of events leading up to a theft or other circumstances that led up to an incident.

External resources

Many industries have professional organizations where practitioners, regardless of their employer, can come together to share information. CSIRT personnel may also be tasked with interfacing with law enforcement and government agencies at times, especially if they are targeted as part of a larger attack perpetrated against a number of similar organizations. Having relationships with external organizations and agencies can assist the CSIRT with intelligence sharing and resources in the event of an incident. These resources include the following:

• High Technology Crime Investigation Association (HTCIA): The HTCIA is an international group of professionals and students with a focus on high-tech crime. Resources include everything from digital forensic techniques to wider enterprise-level information that could aid CSIRT personnel with new techniques and methods. For more information, visit the official website at https://htcia.org/.
• InfraGard: For those CSIRT and information security practitioners in the US, the Federal Bureau of Investigation (FBI) has created a private-public partnership geared toward networking and information sharing. This partnership allows CSIRT members to share information about trends or discuss past investigations. You can find more information at the following website: https://www.infragard.org/.
• Law enforcement: Law enforcement has seen explosive growth in cyber-related criminal activity. In response, a great many law enforcement organizations have increased their capacity to investigate cybercrime. CSIRT leadership should cultivate a relationship with agencies that have cybercrime investigative capabilities. Law enforcement agencies can provide insight into specific threats or crimes being committed and provide CSIRTs with any specific information that concerns them.
• Vendors: External vendors can be leveraged in the event of an incident, and what they can provide is often dependent on the specific line of business (LOB) the organization has engaged them in. For example, an organization’s IPS/IDS solution provider could assist with crafting custom alerting and blocking rules to assist in the detection and containment of malicious activity. Vendors with threat intelligence (TI) capability can also provide guidance on malicious activity indicators. Finally, some organizations will need to engage vendors who have an IR specialty such as reverse engineering malware when those skills fall outside an organization’s capability.

Depending on the size of the organization, it is easy to see how the CSIRT can involve several people. It is critical to putting together the entire CSIRT that each member is aware of their roles and responsibilities. Each member should also be asked for specific guidance on which expertise can be leveraged during the entire IR process. This becomes more important in the next part of the IR framework, which is the creation of an IR plan.

With the IR charter written and the CSIRT formed, the next step is to craft an IR plan. An IR plan is a document that outlines the high-level structure of an organization’s response capability. This is a high-level document that serves as the foundation of the CSIRT. The major components of an IR plan are set out here:

• IR charter: An IR plan should include the mission statement and constituency from the IR charter. This gives the plan continuity between the inception of the IR capability and the IR plan.
• Expanded services catalog: The initial IR charter had general service categories with no real detail, so the IR plan should include specific details of which services the CSIRT will be offering. For example, if forensic services are listed as part of the service offering, the IR plan may state that forensic services include evidence recovery from hard drives, memory forensics, and reverse engineering potentially malicious code in support of an incident. This allows the CSIRT to clearly delineate between a normal request—say, for the searching of a hard drive for an accidentally deleted document not related to an incident, and the imaging of a hard drive in connection with a declared incident.
• CSIRT personnel: As outlined before, there are a great many individuals who comprise the CSIRT. The IR plan will clearly define these roles and responsibilities. Organizations should expand out from just a name and title and define exactly the roles and responsibilities of each individual. It is not advisable to have a turf war during an incident, and having the roles and responsibilities of CSIRT personnel clearly defined goes a long way to reducing this possibility.
• Contact list: An up-to-date contact list should be part of the IR plan. Depending on the organization, the CSIRT may have to respond to an incident 24 hours a day. In this case, the IR plan should have primary and secondary contact information. Organizations can also make use of a rotating on-call CSIRT member who could serve as the first contact in the event of an incident.
• Internal communication plan: Incidents can produce a good deal of chaos as personnel attempt to ascertain what is happening, which resources they need, and who to engage to address the incident. The IR plan internal communication guidance can address this chaos. This portion of the plan addresses the flow of information upward and downward between senior leadership and the CSIRT. Communication sideways between the CSIRT core and support personnel should also be addressed. This limits the individuals who are communicating with each other and cuts down on potentially conflicting instructions.
• Training: The IR plan should also indicate the frequency of training for CSIRT personnel. At a minimum, the entire CSIRT should be put through a tabletop exercise at least annually. In the event that an incident post-mortem analysis indicates a gap in training, that should also be addressed within a reasonable time after the conclusion of the incident.
• Maintenance: Organizations of every size continually change. This can include changes to infrastructure, threats, and personnel. The IR plan should address the frequency of reviews and updates to the IR plan. For example, if the organization acquires another organization, the CSIRT may have to adjust service offerings or incorporate specific individuals and their roles. At a minimum, the IR plan should be updated at least annually. Individual team members should also supplement their skills through individual training and certifications through organizations such as System Administration, Network, and Security (SANS) or on specific digital forensic tools. Organizations can incorporate lessons learned from any exercises conducted into this update.

Incident classification

Not all incidents are equal in their severity and threat to the organization. For example, a virus that infects several computers in a support area of the organization will dictate a different level of response than an active compromise of a critical server. Treating each incident the same will quickly burn out a CSIRT as they will have to respond in the same way to even minor incidents.

As a result, it is important to define within the IR plan an incident classification schema. By classifying incidents and gauging the response, organizations make better use of the CSIRT and ensure that they are not all engaged in minor issues. Here is a sample classification schema:

• High-level incident: A high-level incident is an incident that is expected to cause significant damage, corruption, or loss of critical and/or strategic company or customer information. A high-level incident may involve widespread or extended loss of system or network resources. The event can potentially cause damage to the organization and its corporate public image and result in the organization being liable. Examples of high-level incidents include, but are not limited to, the following:
  • Network intrusion
  • Ransomware
  • Identification of C2 traffic
  • Physical compromise of information systems and compromise of critical information
  • Loss of computer system or removable media containing unencrypted confidential information
  • Widespread and growing malware infection (more than 25% of hosts)
  • Targeted attacks against the IT infrastructure
  • Phishing attacks using the organization’s domain and branding
• Moderate-level incident: A moderate-level incident is an incident that may cause damage, corruption, or loss of replaceable information without compromise (there has been no misuse of sensitive customer information). A moderate-level event may involve significant disruption to a system or network resource. It also may have an impact on the mission of a business unit (BU) within the corporation. Here are some examples of moderate-level incidents:
  • Anticipated or ongoing DoS attack
  • Loss of computer system or removable media containing unencrypted confidential information
  • Misuse or abuse of authorized access; automated intrusion
  • Confined malware infection
  • Unusual system performance or behavior; installation of malicious software
  • Suspicious changes of computer activity
• Low-level incident: A low-level incident is an incident that causes inconvenience and/or unintentional damage or loss of recoverable information. The incident will have little impact on the corporation. Here are some examples of such incidents:
  • Policy or procedural violations detected through compliance reviews or log reviews
  • A lost or stolen laptop or other mobile equipment containing encrypted confidential information
  • Installation of unauthorized software; malware infection of a single PC

One key aspect of the IR plan is the use of playbooks. An IR playbook is a set of instructions and actions to be performed at every step in the IR process. Playbooks are created to give organizations a clear path through the process, but with a degree of flexibility in the event that the incident under investigation does not fit neatly into the box.

A good indicator of which playbooks are critical is the organization’s risk assessment. Examining the risk assessment for any threat rated critical or high will indicate which scenarios need to be addressed via an IR playbook. Most organizations would identify a number of threats—such as a network intrusion via a zero-day exploit, ransomware, or phishing—as critical, requiring preventive and detective controls. As the risk assessment has identified those as critical risks, it is best to start the playbooks with those threats.

In the absence of a risk or threat assessment, organizations should have a minimum of five playbooks that cover the most common scenarios that they will face, as outlined here:

• Phishing
• Malware
• Ransomware
• Vulnerabilities in externally facing systems
• Business email compromise (BEC)

Note

The last several years have demonstrated the devastating impact a ransomware attack can have on an organization. This book will examine several scenarios as part of the overall ransomware threat to provide a better understanding of preparation and response to this type of attack.

For example, let’s examine the breakdown of a playbook for a common threat—social engineering. For this playbook, we are going to divide it out into the IR process that was previously discussed, as follows:

• Preparation: In this section, the organization will highlight the preparation that is undertaken. In the case of phishing, this can include employee awareness to identify potential phishing emails or the use of an email appliance that scans attachments for malware.
• Detection: For phishing attacks, organizations are often alerted by aware employees or through email security controls. Organizations should also plan on receiving alerts via malware prevention or host intrusion prevention system (HIPS) controls.
• Analysis: If an event is detected, analyzing any evidence available will be critical to classifying and appropriately responding to an incident. In this case, the analysis may include examining the compromised host’s memory, examining event logs for suspicious entries, and reviewing any network traffic going to and from the host.
• Containment: If a host has been identified as compromised, it should be isolated from the network.
• Eradication: In the event that malware has been identified, it should be removed. If not, the playbook should have an alternative such as reimaging with a known good image.
• Recovery: The recovery stage includes scanning the host for potential vulnerabilities and monitoring the system for any anomalous traffic.
• Post-incident activity: The playbook should also give guidance on which actions should take place after an incident. Many of these actions will be the same across the catalog of playbooks but are important to include, ensuring that they are completed in full.

The following diagram is a sample playbook for a phishing attack. Note that each phase of the IR cycle is addressed, as well as specific actions that should be taken as part of the response. Additionally, organizations can break specific actions down, such as through log analysis for a certain playbook, for greater detail:

Figure 1.2 – Social engineering playbook

Playbooks can be configured in a number of ways—for example, a written document can be added to the IR plan for specific types of incidents. Other times, organizations can use a flow diagram utilizing software such as iStudio or Visio. They are designed to give the CSIRT and any other personnel a set of instructions to follow in an incident. This allows less time to be wasted if a course of action is planned out. Playbooks serve as a guide, and they should be updated regularly, especially if they are used in an incident, and any key pieces or steps identified. It should be noted that playbooks are not written in stone and are not a checklist. CSIRT personnel are not bound to the playbook in terms of actions and should be free to undertake additional actions if the incident requires it.

Escalation process

A critical component of the IR plan is escalation procedures. Escalation procedures outline who is responsible for moving an event or series of events from just anomalies in the information system to an incident. The CSIRT will become burned out if they are sent to investigate too many false positives. Escalation procedures ensure that the CSIRT is effectively utilized and that personnel is only contacted if their expertise is required.

The procedures start with the parties who are most likely to observe anomalies or events in the system that may be indicative of a larger incident—for example, the help desk may receive a number of calls that indicate a potential malware infection. The escalation procedures may indicate that if malware is detected and cannot be removed via malware prevention controls, personnel are to contact the CSIRT member on call. An important consideration at this step is determining which information should be contained within the escalation report. The following guidelines capture the details necessary for the CSIRT to begin addressing the issue:

• Date and time of detection: This first data point is self-explanatory. There are two main considerations though. The first is the time zone. The preferred time zone is Coordinated Universal Time (UTC). This is especially useful if all systems that are logging activity are configured to UTC. The second is the format. There can be some debate, but a good all-around method is to place the date and time together in the following formation: 20220117T1637 UTC.
• Reporting person: This serves as the initial point of contact for any additional personnel that may be called into the escalation. This individual should have visibility into the incident to answer any questions. This is often the SOC manager or another responsible party.
• Incident type: In the escalation, it is important to identify the type of incident that is being escalated. This may dictate a specific type of response. Here is a list of incident types to consider:
  • Ransomware
  • Malware infection
  • External system compromise
  • Ongoing compromise
  • Data exfiltration
  • C2
  • DoS
  • Other
• Incident severity: This is the first assessment of the incident’s severity. Having an idea of the severity of the issue will allow the IR team to align resources properly.
• The number of systems impacted: If possible, it is important to provide an order of magnitude of the incident. This should include the number of systems, operating system type, and the function of the systems—for example, an incident that is impacting only Linux servers running Apache versus Windows desktops.
• Patient Zero identified: This data is not often included in the initial escalation, as Patient Zero—or the first system identified as compromised—is often located during the analysis phase of an incident.
• Tactics identified: Any tactics that have been identified should be escalated as well. For example, lateral movement tactics using Server Message Block (SMB) or Remote Desktop Protocol (RDP) should be escalated as they indicate a larger network-wide security incident. A good rubric of tactics to use is the MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) framework, which will be covered in a later chapter.
• Indicators of compromise (IOCs): Data points such as IP addresses, domain names, or file hashes related to the initial detection should be passed up to be included as part of the analysis.
• Actions taken: If any actions were taken, the reporting party or their colleagues should be recorded as well. For example, if the reporting group has the ability to block the execution of code and was successfully able to block further execution of malware, this has a direct impact on the type of response the IR team will execute.

There are a variety of methods that can be used to communicate this information to the CSIRT. A ticketing system can be configured to automatically notify CSIRT personnel with a ticket containing pertinent escalation details. Another option is the use of an email template that is sent to specific CSIRT personnel that handles escalations. During an incident, all actions taken by the CSIRT and other personnel from the start of the incident should be documented and tracked.

Information

For organizations that have limited resources and experience a limited number of incidents per year, most IT ticketing systems are sufficient for tracking incidents. The drawback to this method is that these systems generally lack an IR focus and do not have additional features that are designed to support IR activities. Larger organizations that have a higher frequency of incidents may be best served by implementing a purpose-designed IR tracking system. These systems allow the integration of evidence collection and incident playbooks.

CSIRT members will then take control. If they are able to contain malware to that single system and identify an infection vector, they will attempt to remove the malware and, barring that, have the system reimaged and redeployed. At that point, the incident has been successfully concluded. The CSIRT member can document the incident and close it out without having to engage any other resources.

Another example where escalation moves further up into an all-out CSIRT response can start very simply with an audit of Active Directory credentials. In this case, a server administrator with access management responsibilities is conducting a semi-annual audit of administrator credentials. During the audit, they identify three new administrator user accounts that do not tie to any known access rights. After further digging, they determine that these user accounts were created within several hours of each other and were created over a weekend. The server administrator contacts the CSIRT for investigation.

The CSIRT analyst looks at the situation and determines that a compromise may have happened. The CSIRT member directs the server administrator to check event logs for any logins using those administrator accounts. The server administrator identifies two logins: one on a database server and another on a web server in the demilitarized zone (DMZ). The CSIRT analyst then directs the network administrator assigned to the CSIRT to examine network traffic between the SQL database and the web server. Also, based on the circumstances, the CSIRT analyst escalates this to the CSIRT coordinator and informs them of the situation. The CSIRT coordinator then begins the process of engaging other CSIRT core teams and technical support members to assist.

After examining the network traffic, it is determined that an external threat actor has compromised both systems and is in the process of exfiltrating the customer database from the internal network. At this point, the CSIRT coordinator identifies this as a high-level incident and begins the process of bringing support personnel into a briefing. As this incident has involved the compromise of customer data, CSIRT support personnel—such as marketing or communications and legal—need to become involved. If more resources are required, the CSIRT coordinator will take the lead in making that decision.

Escalation procedures are created to ensure that the appropriate individuals have the proper authority and training to call upon resources when needed. The escalation procedures should also address the involvement of other personnel outside the core CSIRT members, based on the severity of the incident. One of the critical functions of escalation procedures is to clearly define which individuals have the authority to declare anomalous activity in an incident. The escalation procedures should also address the involvement of other personnel outside core CSIRT members, based on the severity of the incident.

So far, there have been a number of areas that have been addressed in terms of preparing for an incident. From an initial understanding of the process involved in IR, we moved through the creation of an IR plan and associated playbooks.

Once a capability has been created, it should be run through a table-top exercise to flush out any gaps or deficiencies. This exercise should include a high-level incident scenario that involves the entire team and one of the associated playbooks. A report that details the results of the table-top exercise and any gaps, corrections, or modifications should also be prepared and forwarded to senior leadership. Once leadership has been informed and acknowledges that the CSIRT is ready to deploy, it is now operational.

As the CSIRT becomes comfortable executing the plan under a structured scenario, they may want to try more complex testing measures. Another available option is the Red/Blue or Purple team exercise. This is where the CSIRT is tasked with responding to an authorized penetration test. Here, the team is able to execute against a live adversary and test the plans and playbooks. This significantly increases the value of the penetration test as it provides an insight into the security of the infrastructure as well as the ability of the organization to respond appropriately.

Regardless of the makeup of the team, another key component of CSIRT deployment is the inclusion of regular training. For CSIRT core members, specific training on emerging threats, forensic techniques, and tools should be ongoing. This can be facilitated through third-party training providers or, if available, in-house training. The technical support members of the CSIRT should receive regular training on techniques and tools available.

This is especially important if these members may be called upon during an incident to assist with evidence collection or remediation activities. Finally, other support members should be included in the annual test of the IR plan. Just as with an inaugural test, the organization should pick a high-level incident and work through it using a table-top exercise. Another option for the organization is to marry up the testing of their IR plan with a penetration test. If the organization is able to detect the presence of the penetration test, they have the ability to run through the first phases of the incident and craft a tabletop for the remaining portions.

One final component of the ongoing maintenance of an IR plan is a complete annual review. This annual review is conducted to ensure that any changes in personnel, constituency, or mission that may impact other components of the plan are addressed. In addition to a review of the plan, a complete review of the playbooks is conducted as well. As threats change, it may be necessary to change existing playbooks or add new ones. CSIRT personnel should also feel free to create a new playbook in the event of a new threat emerging. In this way, the CSIRT will be in a better position to address incidents that may impact their organization. Any major changes or additions should also trigger another table-top exercise to validate additional plans and playbooks.

Benjamin Franklin is quoted as saying: “By failing to prepare, you are preparing to fail.” In many ways, this sentiment is quite accurate when it comes to organizations and the threat of cyber attacks. Preparing for a cyber attack is a critical function that must be taken as seriously as any other aspect of cybersecurity. Having a solid understanding of the IR process to build on this with an IR capability can provide organizations with a measure of preparation so that in the event of an incident, they can respond. Keep in mind as we move forward that forensic techniques, TI, and reverse engineering are there to assist an organization to get to the end—that is, back up and running.

This chapter explored some of the preparation that goes into building an IR capability. Selecting a team, creating a plan, and building playbooks and escalation procedures allow a CSIRT to effectively address an incident. The CSIRT and associated plans give structure to the digital forensic techniques to be discussed.

This discussion begins with the next chapter, where proper evidence handling and documentation is the critical first step in investigating an incident.

Test your knowledge by seeing if you can answer the following questions:

1. A table-top exercise should be conducted after changes are made to the IR plan and/or playbooks.
   1. True
   2. False
2. Which of the following roles would not be a member of the CSIRT core team?
   1. Incident coordinator
   2. CSIRT analyst
   3. Legal
3. It is not important to have technical resources available as part of the IR framework to aid during an incident.
   1. True
   2. False
4. A risk assessment is a valid data source for identifying high-risk incidents for playbook creation.
   1. True
   2. False

You can refer to the following resources for more information about what we learned in this chapter:

• Computer Security Incident Handling Guide, NIST SP 800-61 Rev. 2: https://csrc.Nist.gov/publications/detail/sp/800-61/rev-2/final
• European Union Agency for Cybersecurity (ENISA)—Incident Handling in Live Role Playing Handbook: https://www.enisa.europa.eu/topics/trainings-for-cybersecurity-specialists/online-training-material/documents/incident-handling-in-live-role-playing-handbook/view
• Incident Handler’s Handbook by Patrick Kral, SANS Reading Room: https://www.sans.org/reading-room/whitepapers/incident/incident-handlers-handbook-33901