Perhaps it is a coincidence, but, when I looked up the definition of “variant” online the example sentence was about an illness (1). The concept behind variants is appealing; within one model hierarchy, include multiple configurations of your target. But from a testing perspective, well….
If you go to Baskin Robbins ice cream, home of 31 flavors, and ordered a 2 scope cone you would have (31!/(31-2)!) 930 patterns, meaning it would take you almost 18 years to try them all if you did this once per week(2). So if you have an integration model with 8 referenced models, each model having 3 variants; well, just how many weeks do you want to test for?
Related and unrelated variants
To simplify the design and testing process the variants in a model should be related, e.g. if you have a variant for your car, manual or automatic transmission, then a related variant could be for the wave plate (auto) versus clutch (manual) models. An unrelated variant could be for the HVAC system.
Please note, the ice-cream references were supposed to end in the last paragraph, but the “which of these don’t belong” image I found had an ice-cream cone. That was not my intention but it is too late now.
Defining your inclusion matrix
What this matrix shows us (3) is which model variants
- Are allowed with other variants
- Are required by other variants
- Are not impacted by other variants
Validate the matrix
There are two primary methods for validating the inclusion matrix: inlined and pre-computed. With the inlined version, the full set of conditions for a variant to be selected is coded into the variant selection process. With the pre-computed version, the variant logic exists external to the component and the only final value is evaluated in the component.
As you would expect, there are pros and cons to both approaches (4). Having the logic in the component makes it easier for the developer of the module to understand what is going on; however it makes it more difficult for a system developer to have a global view of the variants. On balance the external computation is more likely to provide robust process.
Variants versus reuse
In our first example, an automatic versus manual transmission, this was clearly a variant; the models would be significantly different. But what if we were talking about a 4 speed versus a 5 speed transmission? Could that model, with some refactoring and change of data, be reused?
The key thing to keep in mind is that while a re-used model will require additional tests cases, many of the “basic” tests would already exist and could be modified for the reused data. On the other hand, a model variant will require a whole (or nearly whole) new set of tests.
Why now?
Why is this blog post part of the “impact of COVID” collection, as this is good advice at anytime. The answer is simple: as we social distance, the informal communication that makes poor design (5) tolerable is less prevalent.
This example COVID drives
Adopt this and thrive. (6)
Footnote
- The jest here is that the over use of variants can lead to multiple bugs in your software due to the increased complexity
- In reality this would take far less time as there are some combinations that should never be considered; for instance, anything with coconut.
- The Matrix (the movie) also shows us that Hollywood had (and has) an ill informed conception of what computer programming / hacking really looks like.
- All design is about the pros and cons, there are few things that are 100% “good”
- Informal communication makes that poor design tolerable, but it doesn’t mean that anyone is happy.
- I’m not sure if this is an example of “doggerel” or “bearerel” poetry. Hopefully you can bear with me.