Testing requirements: Part 2

In this final video we are going to tackle the “Must never” and “Must always” cases for requirements. To do so, the use of mathematical analysis is employed to determine if the conditions hold.

Afterword

In some instances “never” and “always” cases cannot be definitively proven; when this happens there are three alternatives. First, subdivide the model into a smaller unit under test and attempt to prove the condition at that level. Second, determine under what conditions the requirement cannot be proven and then implement guards against those conditions. Or finally, explicitly guard against the condition in your model.

Port in a storm

“It is not enough to know how; it’s the when that is key.” With all projects, at some point in the development cycle problems arise.(1) When they do, questions of refactoring or porting should land on your shores. Often simple bug fixes or in-language refactoring is enough to resolve the issue; however when the impact crosses a critical threshold the answer is to port.

Port:
noun: a town or city with a harbor where ships load or unload.
verb: to transfer between two locations or environments

When it’s not right: port(2)

Porting software can be a major endeavor. Prior to this action a 5 part assessment should be made:

  1. Will this software module be in use for the long term(3) (e.g., will it exist in code deployed in the field; what is the life span of that product?)
  2. What is the criticality of the error? Is it: life threatening, product unusable, product behavior degraded, or is it a minor usability issue?
  3. What are the other issues facing the product and their criticality?
  4. If fixed now in-language will a later porting be required?
  5. What is your deadline?

Assuming that the metrics for 1 ~ 4 are high enough, and assuming that there is enough time before your deadline, it is time to port.

Estimating port fees

How do you estimate the time it will take to port your module, e.g. what is your “port fee?”(4) Ask yourself the following questions:

  • How long did it take the first time around; of that time how much was spent on determining what to do versus implementation?
  • Is the new language better suited for the implementation?
  • Is there existing testing infrastructure and can it be repurposed for the new environment?
  • Is the module unique or similar to existing code in the new environment; e.g. how well do you know your new environment?

The last question, how well do you know your environment is key. If you are just starting out in a new environment, the recommendation is to learn on non-critical applications. Learning an environment should never be a high stakes game.

Ah yes, the classic high stakes snakes and ladders game…

Footnotes

  1. Problems can
    • arise: e.g. float up from the buried issues in a poor design
    • fall: e.g. come from outside sources as new information or requirements are given
    • saunter in from the side: e.g. issues of integration with teammates
      All in all, problems are three dimensional in nature
  2. Assuming you are moving forward, port is left.
  3. Stones from pyramids and other monuments have over time found their way into other construction. It is possible that stones from the pictured are part of someone’s house. When determining if a software module will be in use “for the long haul” it is not enough to look at the full systems but all the stones that make it up.
  4. While free trade ports are a hot topic in economics, they do not exist in software.

Testing Requirements Part 1

This is part 4 in my video series on Model-Based Design testing. Today I look at how to test requirements of the form “When X happens the response is Y.” Follow these steps and then you too can enjoy testing.

Post movie credits…

In the video I state there are two basic types of requirements, when X happens Y occurs and X never occurs. In the final video in this series we are going to look at how we can prove the “never occurs” case.