Security-testing code manually

We mentioned that functional testing is just one form of testing. Another important form is security testing. It’s so important and so difficult to get right that it’s typically performed by specialized teams that are separate from traditional QA departments. There are many ways approaches to security testing, but most boil down to one of three categories:

• Inspect source code
• Interact with running code
• Inspect the third-party dependencies used by your project

Also, there are different kinds of problems that security tests can look for. At first glance, some of these problems may not look like they fall under the umbrella of security, but they all contribute to potential data loss or manipulation of your software by malicious actors:

• Non-standard coding practices
• Unsafe coding practices
• Source code dependencies that contain known vulnerabilities

Let’s look at some specific varieties of security testing and see how they use different techniques to look for different sorts of problems.

Static code analysis

You can often find unsafe coding practices simply by asking a security expert to review your source code. For example, if you ask for user input and then use that input to query a database, a wily user might be able to launch a so-called SQL injection attack by including database commands in their input. Competent code reviewers will spot this sort of problem immediately and can often propose easy-to-implement solutions.

For example, the following pseudocode accepts input from the user but doesn’t validate the input before using it in a SQL statement. A clever user could enter a malicious value such as Smith OR (0 = 0) and cause more information to be revealed than the developer intended:

employee_name = get_user_input()
sql = "SELECT salary FROM employee_records WHERE employee_name = $employee_name" ENTERcall_database(sql)

Code reviews can also identify code that might not be obviously unsafe, but that uses non-standard idioms, unusual formatting, or awkward program structure that make code harder for other team members (or even the original author) to read and maintain. This can indirectly make the code more susceptible to security problems in the future, or at the very least make future security problems harder for code reviewers to find.

For example, the following Python function accepts an unusually large number of parameters and then ignores most of them. Both these traits are considered to be poor programming practices, even if neither threatens the behavior or security of the code:

def sum(i, j, k, l, m, n, o, p, q, r):
    return i + j

Static code analysis can sometimes happen automatically. Many IDEs offer static code analysis as a built-in feature: they draw red warning lines under any non-standard or unsafe code they detect. This can be a great help but is best thought of as a complement to manual code reviews rather than a full substitute.

Secret detection

You can think of secret detection as a special form of static code analysis. There are many types of sensitive data that you want to keep out of your software’s source code. It’s not hard to think of examples:

• Passwords
• Deploy keys
• Public SSH or GPG keys
• US Social Security numbers
• Unique personal identification numbers that are used by other countries

Just as static code analysis scans source code to search for programming or security problems, secret detection scans source code to find secrets that should be removed and stored in a more secure location. For example, the following Java code contains a Social Security number that can be seen by anyone with read access to the code:

String bethSSN = "555-12-1212";
if (customerSSN.equals(bethSSN))) {
     System.out.println("Welcome, Beth!");
}

Dynamic analysis

Looking at source code is useful, but there are many categories of software defects that are more easily found by interacting with executing code. This interaction could take the form of using an application’s GUI just like a human would, sending requests to a REST API endpoint, or hitting various URLs of a web app with different values in the requests’ query strings.

For example, your web server might be configured in such a way as to include its version number in the headers of every response. This might seem like harmless information, but it can provide clues to malicious actors about which web server-targeted exploits are likely to work against your site, and which exploits your web server is probably immune to.

To take another example, complicated logic in your code might obscure the fact that you can trigger an unhandled divide-by-zero error by entering a particular set of input values. As discussed earlier, problems like this may not initially feel like security risks, but a clever hacker can often find ways to exploit simple bugs in ways that expose data, cause data loss, or result in denial-of-service attacks.

For example, the following Ruby code could produce a ZeroDivisionError instance when it runs, which, in turn, could cause the program to crash:

puts 'how many hats do you have?'
num_hats = gets.to_i
puts 'how many cats do you have?'
num_cats = gets.to_i
puts "you have #{num_hats / num_cats} hats per cat"

Dependency scanning

Dependency scanning is the practice of comparing the names and version numbers of each of your product’s dependencies against a database of known vulnerabilities and identifying which of those dependencies should be upgraded to a later version or removed entirely to improve your software’s security. Virtually every non-trivial piece of software written these days relies on tens, hundreds, or thousands of third-party, open source libraries. The source code of the most popular libraries is pored over by Black Hat hackers looking for possible exploits. These exploits are often quickly fixed by the library’s maintainers, but if your project is using old, unpatched versions of those libraries, dependency scanning will let you know that your code might be vulnerable to those known exploits.

A perfect example of the need for this type of security scanning is in the news at the time of writing. Many Java projects rely on an open source Java library called Log4j, which provides a convenient way to log informational, warning, or error messages. A vulnerability was recently discovered that allows hackers to remotely run commands or install malware on any computer Log4j is running on. That’s a huge problem! Fortunately, it’s exactly the kind of problem that dependency scanning can spot. Any up-to-date dependency scanner will let you know if your software has a dependency – either directly or via other dependencies – on an unpatched version of Log4j, and will advise you what version of Log4j you should upgrade to.

Container scanning

These days, many software products are delivered as Docker images. The simplest possible description of a Docker image is that it is a Linux distribution that has your application installed on it and is then packaged in an image format that can be executed by Docker or similar tools. If you build a Docker image that includes an out-of-date version of a Linux distribution that contains security vulnerabilities, your application will not be as secure as it could be.

Container scanning looks at the base Linux image that your Dockerized application is installed on and checks a database of known security vulnerabilities to see if your packaged application might be susceptible to exploits. For example, because CentOS 6 stopped being maintained in 2020, the libraries that it includes have many severe security vulnerabilities. Container scanning would alert you to this problem and suggest that you consider upgrading your application’s Docker image to use CentOS 7 or later as a base image.

Manual security testing summary

With that, we’ve looked at a variety of tests designed to detect security vulnerabilities or security-adjacent problems, such as failing to adhere to coding best practices. While it may seem like a lot of different steps to go through before you can put a simple web app into production, there has never been more ways to steal information or shut down a service, and there’s no reason to think that trend will turn around any time soon. So, like it or not, responsible developers need to think about – and probably implement – all these different security tests:

Figure 1.3 – Some of the many types of security testing

Some of these tests must be performed manually. Others have automated tools to help. But automated tests are still burdensome: you still have to install security testing tools or frameworks, configure testing tools, update test frameworks and dependencies, set up and maintain test environments, massage reports, and display reports in some integrated fashion. If you try to simplify matters by outsourcing some of these tasks to outside companies or Software-as-a-Service (SaaS) tools, you’ll need to learn separate GUIs for each tool, maintain different user accounts for each service, manage multiple licenses, and do a host of other tasks to keep your tests working smoothly.

This section has shown you more ways that life before GitLab was difficult for development teams. As you’ll learn in an upcoming chapter, GitLab’s CI/CD pipelines replace the awkward, multi-step security testing processes described previously with fast, automated security scanners that you configure once and then benefit from for as long as you continue to develop your software project. We’ll revisit this topic in much more detail later.