If you have just written a new CICS, JSON/XML service or REST API, you need to perform a load test of the application before you let it loose on the world. Also you need to consider how the system responds when that load gets too high, or stressed. Load or stress testing a component are two different test techniques that often get confused. While discussing this originally A colleague James O’Grady and I came up with an analogy, think of driving a car for a long distance.
A load test is driving the car for 874 miles at an average speed of 60mph, in 5th gear, while using the air conditioning, cruise control and CD player. Using lots of the capabilities of the car at expected limits for a length of time. During, and at the end of the journey, we would expect the car to still be operational, and that all the dials on the dashboard to be reading nominal values. A stress test is a completely different type of test.
In a stress test we want to push the system beyond its limits. Often the limits will not be clear and so often the test becomes exploratory or iterative in nature as the tester is pushing the system toward the limits. If we reuse the driving analogy we might start the same journey but now drive at 70mph in 3rd gear. Initially we think this might be enough to stress the car. After 60 minutes we increase the stress by removing some of the car oil and deflating the tires. By now, some of the dashboard lights are showing us that some of the car components are stressed.
We then remove some of the coolant fluid and remove a spark plug. Now the car is seriously under stress. All the warning lights are on and eventually the car gracefully stops operating and we are forced to steer to the shoulder.
Once safe we re-fill all the fluids and oil, re-inflate the tires and replace the spark plug. Now we are able to re-start the car and resume our journey driving properly.
A stress test means pushing the system beyond the limits it is designed to run at either by restricting resources or by increasing the workload (or often both). This is done until the system either shuts down or restricts further input until it is no longer under stress conditions.
In both cases when testing a full production system it might be difficult to load or stress the entire system using the resources that a test team might have available. Therefore you might wish to artificially construct load or stress scenarios by reducing system resources or constraints to allow the system to enter a loaded state at a much lower throughput. This can allow you to perform this type of testing a lot early in the cycle
In this sequence of posts we will look at one of the tools we use (Jmeter) and how we use this to provide load and stress workloads for CICS.