Preface by Alexander Podelko

An important event, directly related to this book, happened recently and probably went unnoticed. It appears that Apache JMeter has become the most popular load testing tool. In 2014, I was preparing a presentation about load testing tools and the criteria for their selection. One criterion was the existence of an ecosystem (documents, expertise, people, services, and so on). It may be not the defining factor, but it is an important factor to consider. To evaluate such ecosystems, in the absence of more sophisticated data, I used the number of web pages returned by Google and the number of jobs that Monster finds that mention each product.

LoadRunner (then an HP product) clearly held first place in both categories, with JMeter following not too far behind. Silk Performer (then owned by Borland) trailed far behind in third place. But now, in 2018, JMeter appears to be well ahead of LoadRunner (now owned by Micro Focus) in both the number of documents and the number of jobs mentioning it, apparently becoming the most popular load testing tool. Of course, it doesn't mean that JMeter became the best tool for every task, but its popularity, in addition to being an open-source tool, definitely earns it a high place in the list of options to consider. It is also very important for an open-source project to attract people who will work to improve it, thus ensuring the future development of the product.

Another interesting trend is that JMeter scripts have become a de facto standard, and many SaaS tools are built on top of JMeter or at least support JMeter scripts. These tools complement JMeter in many important ways and elevate its functionality and services to a new level, allowing it to compete with commercial products in more sophisticated environments.

Several books about JMeter have been published before, but this one is the first by JMeter contributors who know it inside out and are renowned experts in this area. JMeter is not a trivial product. It has a lot of functionality, but this can be implemented in many different ways and it is not always easy to figure out the best way to do it. While, as we have already mentioned, there is an enormous number of posts on the internet discussing different aspects of JMeter, the problem is that most of them are for beginners. And while there is more advanced stuff too, if you want to use it, you'd better know exactly what you're looking for as it may be hard to find it by searching just for generic terms. This is where this book will be invaluable in establishing a framework of knowledge and familiarity with the basic concepts, so you will at least understand what you should be looking for.

This book, with all the aforementioned valuable information in one place, is a must for anybody who is working seriously with JMeter. It may be a little too condensed for absolute beginners (although, as already mentioned, there are lots of introductory materials on the internet that can help here), but it is the best resource that you can find for JMeter if you already have some performance-testing experience and need to go further. In particular, it will be invaluable to people who want to expand their JMeter knowledge into advanced topics or switch to JMeter from other load testing tools.

In particular, JMeter has many integrations with other popular tools and a large number of plugins. It is quite possible that somebody has already solved the problem you are confronting, so you may save a lot of effort by simply reusing their solution. It is fortunate that this book is not limited to the core JMeter functionality and discusses available components and integrations when appropriate. While it is impossible to cover everything that is available in detail, the advantage of this book is that it puts everything into a system, allowing the reader to understand the relations between different parts and technologies.

Load testing is an important part of the performance-engineering process. However, the industry is changing and load testing needs to adjust to these changes. A stereotypical, last-minute performance check is not enough anymore. Performance testing should be interwoven into the development process, starting early and continuing throughout the whole life cycle of the product. The importance of this transformation is stressed in the title of the book itself, From Load Testing to DevOps, moving from standalone load testing as a mere step at the end of the software development cycle, to performance testing fully integrated into DevOps. And while we are not fully there yet, the Chapter 13, Integration of JMeter in the DevOps Tool Chain dives directly into what can be done right now.

DevOps, putting together the development and operations sides, is supposed to drastically improve feedback from production to development, and the free flow of performance information in both directions. So, a holistic approach to performance should be one of its main advantages. Unfortunately, it doesn't look like such a holistic approach happens often. Rather, it looks like DevOps teams just drop the more sophisticated parts of performance engineering (and performance testing usually gets into that category) and rely on a more reactive approach to performance issues, concentrating more on quick fixes for issues than on their prevention. Still, load testing is a very important method of risk mitigation and can't be fully replaced by other performance-engineering activities. Let's have a quick look at the benefits delivered by load testing.

First, there are always the risks of crashing a system or experiencing performance issues under heavy load, and the only way to mitigate them is to actually test the system. Even stellar performance in production and a highly scalable architecture don't guarantee that it won't crash under a slightly heavier load.

It is important to note that load testing doesn't completely guarantee that the system won't crash: this would be the case if, for example, the real-life workload was different from what was tested. So, you need to monitor the production system to verify that your test workload is close enough. That said, load tests do considerably reduce the risk if they are carried out correctly (and, of course, can be completely useless and misleading if they are not).

Another important benefit of load testing is in verifying how changes affect multi-user performance. The impact of changes on multi-user performance is generally not proportional to what is observed with single-user performance and can often be counterintuitive. Sometimes, improving single-user performance can lead to a degradation of multiuser performance. The more complex the system, the more exotic the multiuser performance problems that can occur are.

Another value of load testing is to provide a reliable and reproducible way to apply a multiuser load necessary for performance optimization and troubleshooting. You apply exactly the same workload and see if the change makes a difference. In most cases, you cannot do this in production when the workload changes, so you never know if the result is due to a change in the code or a change in the workload (except, perhaps, the rather rare case of very homogeneous and manageable workloads, where you can apply a very precisely measured portion of the actual workload). And, of course, a reproducible workload greatly simplifies the debugging and checking of multiuser problems.

In addition, given current trends in system self-regulation (such as auto-scaling or load-dependent service level changes), load tests are required to verify this functionality. You must apply a heavy load to see how auto-scaling will work. Load testing thus becomes a means of testing the functionality of the system, blurring the traditional division between functional and non-functional testing.

You will find further examples of different types of performance tests and their links to different aspects of DevOps in the book. Although the book does not focus on the theoretical aspects of performance testing, it provides sufficient theoretical information to understand the concepts discussed and their practical applications in JMeter. Practical examples of integrating performance testing with DevOps are all the more important as this is a rather new field of expertise and is probably the main challenge of performance testing at the moment.

This book leads the reader through the basics of working with today's most popular load testing tool, JMeter, through more advanced aspects of the tool, and performance testing in general, through to its complete integration into DevOps.

Alexander Podelko Bio

Over the last 20 years, Alex Podelko has supported major performance initiatives for Oracle, Hyperion, Aetna, and Intel in different roles, including as a performance tester, a performance analyst, a performance architect, and a performance engineer. Currently, he is a consulting member of technical staff at Oracle, responsible for the performance testing and tuning of Hyperion (that is, Enterprise Performance Management and Business Intelligence) products. Before specializing in performance, Alex led software development for Rodnik Software. Having more than 30 years of overall experience in the software industry, he holds a Ph.D. in computer science from Gubkin University and an MBA from Bellevue University.

Alex periodically talks and writes about performance-related topics, advocating tearing down silo walls between different groups of performance professionals. He currently serves as a board director for the Computer Measurement Group (CMG), a worldwide organization of performance and capacity management professionals.