Performance testing and tuning
There is a strong relationship between performance testing and tuning, in the sense that one often leads to the other. Often, end-to-end testing unveils system or application bottlenecks that are regarded as incompatible with project target goals. Once those bottlenecks are discovered, the next step for most teams is a series of tuning efforts to make the application perform adequately.
Such efforts normally include but are not limited to:
Configuring changes in system resources
Optimizing database queries
Reducing round trips in application calls; sometimes leading to re-designing and re-architecting problematic modules
Scaling out application and database server capacity
Reducing application resource footprint
Optimizing and refactoring code; including eliminating redundancy, and reducing execution time
Tuning efforts may also commence if the application has reached acceptable performance but the team wants to reduce the amount of system resources being used, decrease volume of hardware needed, or further increase system performance.
After each change (or series of changes), the test is re-executed to see whether performance has increased or declined as a result of the changes. The process will be continued until the performance results reach acceptable goals. The outcome of these test-tuning circles normally produces a baseline.
Baselines
Baseline is a process of capturing performance metric data for the sole purpose of evaluating the efficacy of successive changes to the system or application. It is important that all characteristics and configurations except those specifically being varied for comparison remain the same, in order to make effective comparisons as to which change (or series of changes) is the driving result towards the targeted goal. Armed with such baseline results, subsequent changes can be made to system configuration or application and testing results compared to see whether such changes were relevant or not. Some considerations when generating baselines include:
They are application specific
They can be created for system, application, or modules
They are metrics/results
They should not be over generalized
They evolve and may need to be redefined from time to time
They act as a shared frame of reference
They are reusable
They help identify changes in performance
Load and stress testing
Load testing is the process of putting demand on a system and measuring its response; that is, determining how much volume the system can handle. Stress testing is the process of subjecting the system to unusually high loads far beyond its normal usage pattern to determine its responsiveness. These are different from performance testing whose sole purpose is to determine the response and effectiveness of a system; that is, how fast is the system. Since load ultimately affects how a system responds, performance testing is almost always done in conjunction with stress testing.
