Testing is like hand washing

Prior to the start of sheltering-in place, with the news of COVID-19 dominating the news I saw a marked uptick in the number of paper towels showing up, often on the floor due to overflow, of the bathroom at work. While on the one (cleaner) hand, I was happy to see an increase in hand washing, on the other hand it drew attention to the fact that people had not been washing their hands regularly. This is a common human behavior; we respond to a problem by starting what we should have been doing all along, e.g. “Dentist” Flossing.(1)

Emergency!!! Start testing

Frequently testing starts in response to a major issue, the ‘oh #@$#%!” moment. Then, a few weeks or months later, the support for testing dies down when the problem is passed. There are problems with this

Testing done after an emergency tends to test against the emergency condition; e.g. there isn’t a generic response to improve test coverage across the board. (2)
It is created by inexperienced test developers. Writing good tests requires a knowledge of testing best practices
It is done without supporting infrastructure; as a result the tests are both more complicated to write and more likely to fail.

It is too late now! Or is it?

Even companies with existing test automation are seeing their infrastructure stress tested by the increased demands placed on it by remote workers. There are 3 things that can be done to improve the results in the short term and “harden” your infrastructure for the long term

Create basic test patterns (hook) for end users: If everyone writes their own testing patterns (which is common) the system cannot integrate all the different methods used.
- Keep the interface simple for the end user
- Give them flexibility for what is inside the test
- Provide the error handling (crash protection) for them.
Set up a test review group: There should be both automatic checking that the tests submitted conform to the standard and a review group that validates that the tests are covering what is described.
Leverage existing technologies and workflows for cloud based testing: Use of Jenkins or MUNITs take advantage of the wealth of existing tools and processes that exist for testing in the model based design environment.

Final thoughts

The best testing is formalized and repeatable and easy (for the end user) to understand how it is used. Most importantly, over time, the use of the testing infrastructure becomes habit, and that is how we keep “bugs” from spreading.

Footnotes

Dentist flossing: people who don’t floss normally but start flossing furiously the week before a visit to the dentist.
A narrow focus on fixing a bug may not get to the root cause of the problem. While the goal should be to fix the existing issue the time spent in root causing major issues is, almost always worth spending.