Performance testing and tuning

There is a strong relationship between performance testing and tuning, in the sense that one often leads to the other. Often, end-to-end testing unveils system or application bottlenecks that are regarded unacceptable with project target goals. Once those bottlenecks are discovered, the next steps for most teams are a series of tuning efforts to make the application perform adequately.

Such efforts are normally included but are not limited to:

• Configuring changes in system resources
• Optimizing database queries
• Reducing round trips in application calls; sometimes leading to redesigning and re-architecting problematic modules
• Scaling out application and database server capacity
• Reducing application resource footprint
• Optimizing and refactoring code, including eliminating redundancy and reducing execution time

Tuning efforts may also commence if the application has reached acceptable performance but the team wants to reduce the amount of system resources being used, decrease volume of hardware needed, or further increase in system performance.

After each change (or series of changes), the test is re-executed to see whether performance has improved or declined as a result of the changes. The process will be continued with the performance results having reached acceptable goals. The outcome of these test-tuning circles normally produces a baseline.

Baselines

Baseline is a process of capturing performance metric data for the sole purpose of evaluating the efficacy of successive changes to the system or application. It is important that all characteristics and configurations except those specifically being varied for comparison remain the same in order to make effective comparisons as to which change (or series of changes) is driving results towards the targeted goal. Armed with such baseline results, subsequent changes can be made to the system configuration or application and testing results can be compared to see whether such changes were relevant or not. Some considerations when generating baselines include:

• They are application-specific
• They can be created for system, application, or modules
• They are metrics/results
• They should not be over generalized
• They evolve and may need to be redefined from time to time
• They act as a shared frame of reference
• They are reusable
• They help identify changes in performance

Load and stress testing

Load testing is the process of putting demand on a system and measuring its response, that is, determining how much volume the system can handle. Stress testing is the process of subjecting the system to unusually high loads far beyond its normal usage pattern to determine its responsiveness. These are different from performance testing whose sole purpose is to determine the response and effectiveness of a system, that is, how fast the system is. Since load ultimately affects how a system responds, performance testing is almost always done in conjunction with stress testing.