Friday, 24 April 2015

Difference between load and stress - using a metaphor

Load or stress testing a component are two different test techniques that often get confused.  Here is an analogy which I have modified from a conversation I had with James O'Grady

A load test is driving the car for 874 miles at an average speed of 60mph, in 5th gear, while using the air conditioning, cruise control and CD player.  Using lots of the capabilities of the car at expected limits for a length of time.  During and at the end of the journey we would expect the car to still be operational and all the dials on the dashboard to be reading nominal values.  A stress test is a completely different type of test.

In a stress test we want to push the system beyond its limits.  Often the limits will not be clear and so often the test becomes exploratory or iterative in nature as the tester is pushing the system toward the limits.  If we reuse the driving analogy we might start the same journey but now drive at 70mph in 3rd gear.  Initially we think this might be enough to stress the car.  After 60 minutes we increase the stress by removing some of the car oil and deflating the tyres.  Now some of the dashboard lights are showing us that some of the car components are stressed.  We then remove some of the coolant fluid and remove a spark plug.  Now the car is seriously under stress.  All the lights are on and eventually the car gracefully stops operating and we are forced to steer to the hard shoulder.  Once safe we re-fill all the fluid and oil, re-inflate the tyres and repair the spark plug.  Now we are able to restart the car and resume our journey driving properly.

A stress test is pushing the system beyond the limits it is designed to run at either by restricting resources or by increasing the workload (or often both).  This is done until the system either gracefully shuts down or restricts further input until it is now longer under stress conditions.

Both tests are heavily contextual as it relies on a deep understanding on how the software will be used in the wild.  Will a customer use the software for a long period of time under a load condition or do they just use it in short bursts.  This question is more important when you consider software built in the cloud.

If your software is built in the cloud and you are re-deploying every 2 weeks then your view of load and stress testing will be different to testing an on prem application as the operational realities of using that software are contextually different.

No comments: