In the last section, we covered the fundamentals of conducting a performance test. One of the areas performance testing covers is testing tools. Which testing tool do you use to put the system and application under load? There are numerous testing tools available to perform this operation, from free to commercial solutions. However, our focus in this book will be on Apache JMeter, a free, open source, cross-platform desktop application from the Apache Software foundation. JMeter has been around since 1998 according to historic change logs on its official site, making it a mature, robust, and reliable testing tool. Cost may also have played a role in its wide adoption. Small companies usually don't want to foot the bill for commercial end testing tools, which often place restrictions, for example, on how many concurrent users one can spin off. My first encounter with JMeter was exactly a result of this. I worked in a small shop that had paid for a commercial testing tool, but during the course of testing, we had outrun the licensing limits of how many concurrent users we needed to simulate for realistic test plans. Since JMeter was free, we explored it and were quite delighted with the offerings and the share amount of features we got for free.
Here are some of its features:
- Performance tests of different server types, including web (HTTP and HTTPS), SOAP, database, LDAP, JMS, mail, and native commands or shell scripts
- Complete portability across various operating systems
- Full multithreading framework allowing concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups
- Full featured Test IDE that allows fast test plan recording, building, and debugging
- Dashboard report for detailed analysis of application performance indexes and key transactions
- In-built integration with real-time reporting and analysis tools, such as Graphite, InfluxDB, and Grafana, to name a few
- Complete dynamic HTML reports
- Graphical User Interface (GUI)
- HTTP proxy recording server
- Caching and offline analysis/replaying of test results
- High extensibility
- Live view of results as testing is being conducted
JMeter allows multiple concurrent users to be simulated on the application, allowing you to work toward most of the target goals obtained earlier in this chapter, such as attaining baselines and identifying bottlenecks.
It will help answer questions, such as the following:
- Will the application still be responsive if 50 users are accessing it concurrently?
- How reliable will it be under a load of 200 users?
- How much of the system resources will be consumed under a load of 250 users?
- What will the throughput look like with 1000 users active in the system?
- What will be the response time for the various components in the application under load?
JMeter, however, should not be confused with a browser (more on this in Chapter 2, Recording Your First Test, and Chapter 3, Submitting Forms). It doesn't perform all the operations supported by browsers; in particular, JMeter does not execute JavaScript found in HTML pages, nor does it render HTML pages the way a browser does. However, it does give you the ability to view request responses as HTML through many of its listeners, but the timings are not included in any samples. Furthermore, there are limitations to how many users can be spun on a single machine. These vary depending on the machine specifications (for example, memory, processor speed, and so on) and the test scenarios being executed. In our experience, we have mostly been able to successfully spin off 250-450 users on a single machine with a 2.2 GHz processor and 8 GB of RAM.