In praise, in damnation of the simple example

Faust &
Mephistopheles 

When I was first learning German language in high school the first book(1) that we read was “Emil und die Detektive” a “young adult” book.

Wann ich ersten Deutsche studiere, ich lessen die Buch “Emil and the detectives”

The final book that I read was Goethe’s Faust. Needless to say the later book was a of more interest. I remember the ideas of Faust, I remember little of the Detectives, so why did I start with YA lit?(2) And why am I writing about it a blog about Model-Based Design?

Learning fundamentals

When learning a language it is obvious that you should start with simple examples where you learn the fundamentals of the structure and expand your vocabulary. However if you never move to more complex subjects your understanding of the language (and in the case of literature life itself) will be forever arrested at a low level.

The gain block: Hello world of Simulink

The cliche first program is often the simple print out of “hello world”. In Simulink the first model is “Input, gain, output”. It can be used to show

  1. How blocks are added to the model
  2. How blocks are connected
  3. How to parameterize the model
  4. How to simulate a model….

After learning, simple models

Image result for simple models

Finally on to the subject of this post; why do we create simple models once we have learned the “language”?

  1. To validate understanding: for blocks / constructs that are more complex a simple model that exercises range of behaviors helps us understand it under all modes of operation. 
  2. Debugging: You are trying to isolate parts of the model to determine where an error in you system occurs.
  3. Instructional: You are using the model to introduce people to basic concepts

Why models are not like books…

When I read a YA book in German I knew, through daily life, that the language of the book was targeted at younger readers. When I see an example model in a new language I don’t know if the pattern I’m seeing is the “adult” pattern or the YA pattern.

The most common “faults” of example models are(3)

  • Size: large models serve no purpose in showing concept basic concepts.
    • Mitigation: create example “large model” cases.  Walk through the best practices for large models (and system of models.
  • Efficiency: to illustrate one concept we will often make other parts of the model simpler.  
    • Mitigation: explicitly state the area of the model that is expanded for instruction
  • Problem specific: sometimes examples are intended to demonstrate how to solve a specific problem but that solution maybe incorrect in other situations.
    • Mitigation: explicitly state what the scope of the solution is
  • Lack of domain knowledge: this last issue is the hardest to deal with; on occasion example models are created by people who are new to the language and, as a result, they come up with solutions that are non-advisable.
    • Mitigation: peer review and look to industry sources for examples.

Footnotes

  1. The second book was “The little prince”; which in hindsight was an odd choice since it was originally written in French.  Still I could never forget the word for sunrise and sunset.  (Sonnenuntergang und Sonnenaufgang; this makes singing songs from Fiddler on the Roof in German difficult)
  2. In truth, written in 1928, it provides a much more realistic view of life than modern Young Adult novels. Danke, dass Sie Goss vermissen.
  3. In all honesty I have been guilty of all of these things in my examples over the years; however I have tried to correct this through the mitigation recommendations I have listed.

Common/uncommon faults

“Z key holder”

Last week the “Z” key popped off of my laptop. As my standard language of communication is English this has not been a big deal; however on Friday when I went to write to an email in German things changed. More correctly the task (typing) was the same but the mode (language) changed. Never one to let a random thing pass I thought “wow this is a great topic for a blog post”.(1)

Testing in the “comfort zone”

Your comfort zone

It is a well known problem with testing that we tend to test things that we know the best, as a result we end up with incomplete test coverage. There are two problems, first the obvious one that we are not covering everything. Second we are wasting time by creating redundant tests in the “well known” areas.(2)

How to exit your comfort zone?

Let’s look at three types of models, mode based, continuous (e.g. equation), logical and hybrid(3).

  • Mode base: existing the “CZ” with mode based models is simple; validate that you are testing every mode in the model.
    • It should be noted that with Mode base models there is often a temporal component that must be validated.
    • Additionally the “mode to mode” paths should be validated as often initial conditions can be different depending on which mode you have transitioned from.
  • Continuous (equation): these models can be fully tested by exercising the full range of all inputs. For example, let say you have two inputs U1 and U2 with ranges [0 to 10] and [-5 to 5]. The test vectors would cover a range [0,-5: 10,5]. There are a few considerations.
    • Test spacing: depending on how sensitive the output is to changes in the input the “steps” in the coverage needs to be adjusted. E.g. could you “step” inputs of U1 in 0.1 or 0.5 increments?
    • Exclusivity: in some instances inputs are related, e.g. the temperature of your car’s engine is never less than the outside temperature.(4) This can reduce the test range
    • Temporal: another factor is “how long” are you at each data point?
  • Logical: are similar to mode based testing, however they lack the state information that mode based testing implies. Like mode based testing this is validated by exercising each logical path in the model. Tools like Simulink Design Verifier can be used to generate these test vectors.
  • Hybrid: E.g. 95% of all models. This is where design of experiments comes into play. For large systems it may not be practical to test “every point”. However that is not the goal of testing, the objective of testing is to test every operational behavior.

The upshot

Testing = Hacking(6)

Good tests take time; both to develop and to execute. Assuming a library of basic test functionality and a well written requirement document for the component you can estimate the number of test “points” as a function of the number of modes, inputs and logical branches.
TP = 1.25 * Modes + (min(0,numInputs/2) * numOutputs)^1.5 + numLogical/2; (5)

This formula empirical, and is derived from a review of test sets for well ordered models. The assumptions built into the formula are

  1. The “Mode-to-mode” connections are limited, e.g. not every mode can transition to every other mode
  2. There is a fair degree of mutual exclusivity in the input vectors.
  3. The number of tests is more sensitive to the number of outputs than number of inputs.
  4. Logical tests can often have redundant paths due to the lack of state information.

The final part of this equation is “time of construction”. Time of construction refers to how long it takes to create each test point. Both mode based and logic based tests vectors can be automatically generated, often achieving 100% coverage. (Or showing that 100% coverage is not possible and that there is an error in your logic). As a result I generally estimate the time to develop these tests as

t = (NumModes + NumLogic) * 1 minute;

The time assumes that some level of errors in modeling will be discovered and corrected. For the equation (continuous) testing the time is dependent on the coverage of the requirements; e.g. the more of the input space that the requirements cover the lower the total testing times.

t = Num_Req * 45 minutes + (10 minutes * %notCovered * 100)

Again, this is an empirical formula based on the well ordered models and an existing testing infrastructure.

Footnotes

  1. This blog post will be forwarded to my manager to explain why there is a $23 repair bill sent in
  2. I was once proudly told by an engineer they had over a 100 tests for their system; the problem was that those 100 tests were all dedicated to 6 out of the 27 functions in the system. We corrected that issue.
  3. That sentence should have been tested, as there are 4 “types” in there, not 3. This is what happens when you “design” a sentence with conceived notions.
  4. Ok, that statement isn’t absolutely true, if you had your car in a “cold box” and then drove it out into a warm day for a short period of time that would be true. At the same time if you are storing your care in a “cold box” you
  5. Hmmm, ending with a “;” shows just how long I have been programing in MATLAB.
  6. Again, thank you https://smbc-comics.com/

Please consider subscribing for regular updates