Testing the basics

As stated by Boris Beizer, author of the book Software Testing Techniques, Dreamtech Press:

"Bugs lurk in corners and congregate at boundaries."

Security testing can be defined as a process through which we find vulnerabilities or flaws in our security system. Although we may do exhaustive security testing, it does not imply that no flaws exist. In this section, we will focus on the taxonomy of tests that can be performed in any circumstance.

Tests can be categorized into two big groups: white-box tests or structural tests and black-box tests or functional tests. Structural testing, more commonly known as the white-box testing, is a testing method that evaluates the internal behavior of a component. It is focused on the analysis of the behavior of each procedure in different moments of execution. The white-box test evaluates how the software produces a result. Functional testing, specification testing, or black-box testing, are methods of testing that focus on the functionality of the component rather than its structure. When using this kind of test, the tester is aware that a certain input should generate a particular output. This test evaluates what the software produces.

The two test categories, white-box test and black-box test, are shown in the following diagrams:

There are various white-box techniques. However, the most commonly used are control flow testing, data flow testing, basis path testing, and statement coverage and they are explained as follows:

• Control flow testing: This evaluates the flow graph of the software to indicate whether the set of tests covers every possible test case.
• Data flow testing: This requires an evaluation of how the program variables are used.
• Basis path testing: This ensures that every possible path in a code has been included in the test cases.
• Statement coverage: This consists of the evaluation of the code and the development of individual tests that will work on every individual line of code.

The black-box testing design also includes different techniques. The most frequently used techniques are equivalence partitioning, boundary value analysis, cause-effect graphing, state transition testing, all pairs testing, and syntax testing, and they are explained as follows:

• Equivalence partitioning: This divides test cases in different partitions that present similar characteristics. This technique can help in reducing the number of tests cases.
• Boundary value analysis: This is performed in order to analyze the behavior of a component when the input is near the extreme valid values.
• Cause-effect graphing: This graphically illustrates the relationship between circumstances or events that cause a determined effect on the system.
• State transition testing: This is performed through a number of inputs that make the system execute valid or invalid state transitions.
• All pairs testing: This is a combinatorial method that tests every possible combination of parameters. When the number of parameters and the possible values for each parameter are big, this test technique can be combined with the equivalent partitioning technique to reduce the number of test cases.
• Syntax testing: This analyses the specifications of a component to evaluate its behavior with a huge number of different inputs. This process is usually automatized due to the large number of inputs required.

When testing an application, there are different levels of testing that depend on the size of the part of the system involved. There are five commonly known levels of tests: unit, integration, validation, system, and acceptance.

• Unit tests: These tests focus on each individual component. These tests are usually performed by the same development team and consist of a series of tests that evaluate the behavior of a single component checking for the correctness of the data and its integrity.
• Integration tests: These tests are performed by the development team. These tests assess the communication between different components.
• Validation tests: These tests are performed by the fully developed software in order to evaluate the fulfilment of functional and performance requirements. They can also be used to assess how easy it is to maintain or to see how the software manages errors.
• System tests: These tests involve the whole system. Once the software is validated, it is integrated in the system.
• Acceptance tests: These tests are performed in the real environment where the software is used. The user performs these tests and accepts the final product.

The higher the level of testing, unit testing being the lowest and acceptance testing the highest, the more likely it is to use black-box tests. Unit tests evaluate components that are small and therefore easy to analyze in behavior. However, the higher the level, the bigger the system, and therefore the more difficult and more resource-consuming it is to apply white-box testing category. This does not mean that you should not apply the black-box testing category while performing unit tests, as each one complements the other.