Model-Based Design

I have drawn the software design “V” roughly 1e04 times.(1) Over time, the scope of what is in the V for Model-Based Design has increased. Sometimes I think there should be a Model-Based Design equivalent to Moore’s Law (4); perhaps the “delta V”.(5)

In the arms of the V

Often when asked “what’s next?”, deeper is the answer. Improve a product, improve the process, increase (or decrease) the outputs; the less obvious answer is what can be brought into the embrace of the V.

As the scope of Model-Based Design expands, it has done so in a technology first, workflow second hodgepodge.(6) Uniquely what we are now seeing is the embrace of model based systems engineering which focuses on workflow first and tools second.

What is a system?

Definitions of systems are generally unsatisfying as there is not consensus on what comprises a system; however the definition I like best is…

Open and Closed Systems: A system is commonly defined as a group of interacting units or elements that have a common purpose. … The boundaries of open systems, because they interact with other systems or environments, are more flexible than those of closed systems, which are rigid, and largely impenetrable

Come together, right now, over???

I doubt the Beatles were thinking of software systems when they wrote this song (7) but it gets to the heart of why systems are important. They give people a place to “come together.” No model is, or should be, an island.(8) But how do we come together? The answer is through abstraction.

How does the centipede walk? (9)

The objective of a systems integration environment is to provide an abstracted language so that software, hardware, controls, physical modelers… can all exchange “information” without having to understand the heart of each other’s domain.

At a minimum, a system acts as both a “living” ICD document and as an ICD verification tool. When working in the metadata rich Model-Based Design environment a system integration tool can do much more.

Used well, metadata is an asset, used poorly it can sink your ship.(10) Use of a systems integration tool provides a natural “organizational” layer on top of the metadata. System interactions, data dependencies, system wide requirement coverage all can be accessed from within a system level model.

The Worm Ouroboros: Model-Based Design or Model Based System Engineering

I’ve been asked “What is the boundary between MBD and MBSE”? Right now where it lies is an open question (11) and not at present a fruitful question. Five years ago there was a clear answer, and five years from now we will again have clarity, but right now we are in a land rush (12) where developers can explore new ideas and stake out new domains. What is next? A better understanding of what is.

Footnotes

1. I came to this number using a Fermi Estimation methodology.
   Years in MBD Community ~ 20, Customer per year ~ 50, Drawings per customer ~ 10
   Years mentoring in MBD ~ 10, mentored per year 4, drawings per mentored ~ 20 (2)
   tot = 20 * 50 * 10 + 10 * 4 * 20 = 10,800. (3)
2. In the end, the mentored end up drawing it and having their own way of talking about the design V.
3. The Fermi estimate may make you think that the 800 from mentoring is not important; however, if I count the number of times the people I taught in turn draw this, then I may hit another order of magnitude.
4. It always seemed that Moore’s law should be called “Moore’s extrapolation.”
5. You could think of each software release as an instantaneous acceleration.
6. Technology first, workflow second: a tool for solving a specific problem is developed; how that tool fits into the overall workflow comes later (along with refinements to the tool).
7. And I doubt it was an endorsement of open source software.
8. John Donne poem “No man is an Island” : (with apologies)

No man is an island entire of itself; every man
is a piece of the continent, a part of the main;

No block is a model entire of itself; every block
is a peace of a subsystem, a call from the main();

This post is a companion blog to the earlier post on reuse. Here I will demonstrate how to generate scenario-based test vectors. Enjoy!

Perhaps it is a coincidence, but, when I looked up the definition of “variant” online the example sentence was about an illness (1). The concept behind variants is appealing; within one model hierarchy, include multiple configurations of your target. But from a testing perspective, well….

If you go to Baskin Robbins ice cream, home of 31 flavors, and ordered a 2 scope cone you would have (31!/(31-2)!) 930 patterns, meaning it would take you almost 18 years to try them all if you did this once per week(2). So if you have an integration model with 8 referenced models, each model having 3 variants; well, just how many weeks do you want to test for?

Related and unrelated variants

To simplify the design and testing process the variants in a model should be related, e.g. if you have a variant for your car, manual or automatic transmission, then a related variant could be for the wave plate (auto) versus clutch (manual) models. An unrelated variant could be for the HVAC system.

Please note, the ice-cream references were supposed to end in the last paragraph, but the “which of these don’t belong” image I found had an ice-cream cone. That was not my intention but it is too late now.

Defining your inclusion matrix

What this matrix shows us (3) is which model variants

• Are allowed with other variants
• Are required by other variants
• Are not impacted by other variants

Validate the matrix

There are two primary methods for validating the inclusion matrix: inlined and pre-computed. With the inlined version, the full set of conditions for a variant to be selected is coded into the variant selection process. With the pre-computed version, the variant logic exists external to the component and the only final value is evaluated in the component.

As you would expect, there are pros and cons to both approaches (4). Having the logic in the component makes it easier for the developer of the module to understand what is going on; however it makes it more difficult for a system developer to have a global view of the variants. On balance the external computation is more likely to provide robust process.

Variants versus reuse

In our first example, an automatic versus manual transmission, this was clearly a variant; the models would be significantly different. But what if we were talking about a 4 speed versus a 5 speed transmission? Could that model, with some refactoring and change of data, be reused?

The key thing to keep in mind is that while a re-used model will require additional tests cases, many of the “basic” tests would already exist and could be modified for the reused data. On the other hand, a model variant will require a whole (or nearly whole) new set of tests.

Why now?

Why is this blog post part of the “impact of COVID” collection, as this is good advice at anytime. The answer is simple: as we social distance, the informal communication that makes poor design (5) tolerable is less prevalent.

This example COVID drives
Adopt this and thrive. (6)

Footnote

1. The jest here is that the over use of variants can lead to multiple bugs in your software due to the increased complexity
2. In reality this would take far less time as there are some combinations that should never be considered; for instance, anything with coconut.
3. The Matrix (the movie) also shows us that Hollywood had (and has) an ill informed conception of what computer programming / hacking really looks like.
4. All design is about the pros and cons, there are few things that are 100% “good”
5. Informal communication makes that poor design tolerable, but it doesn’t mean that anyone is happy.
6. I’m not sure if this is an example of “doggerel” or “bearerel” poetry. Hopefully you can bear with me.

When it comes to safety standards, such as the ISO-26262, the old adage “better late than never” can be both dangerous (1) and costly. The simplest, somewhat humorous, description of a safety critical standard that I have read is the following

• Say what you are going to do
• Do what you said you would do
• Verify that what you did matches what you said
• Generate reports

While it is fictitious in its simplicity, it gets to the heart of the matter. Safety critical processes are about being able to show that each step along the development path you both plan out what you intend to do and verify that what you did matches your intentions (plan, do, show).

If that is all

You have to do, how hard can it be? Well first, let’s talk about what it takes to “show” that you did what you said. First you need to be able to show traceability between all artifacts. This means you need a robust tool that will show the link between

• Requirements to the model
• Requirements to tests
• Test results to the model
• Test results to the requirements
• The model to the generated code
• Integration tests to requirements
• And between all the other components

Furthermore those links need to take into account the version control for all the units under test. Setting up the “hooks”(2) for all of these components is a task that needs to be done at the start of the project. There is a reason for the old joke about airplane development: “For every pound of plan you have 20 pounds of paper”(3)

Now that you have your “outline” what now(4)

Tracing the steps is just the “first” step; next, you need to validate the behavior of every tool (including your tracing tools) along the way. There are 5 basic steps for validating a software tool

1. Create a validation plan: Define what it is you will be testing, under what conditions, what is the environment, who will perform the validation…
2. Define system requirements: The creation of a system requirements (SRS) document breaks down into two parts; infrastructural and functional. During this time a system risk analysis document is created along with mitigation strategies.
3. Create a validation protocol and test specs: Definition of both test plan (how you will test) as well as the specific test cases. The creation of a traceability(3) matrix between the plan and the test plan is created.
4. Perform the testing: Execution of the tests defined in step 3
5. Review and update: Collect the results of step 4, for any issues that failed the validation plan, determine if the fall under the mitigation strategies or if the plan / tool needs to be updated.

Getting your ducks in a row

Assuming you get all your ducks in a row (6) what next? The next step is to roll out how you will use those tools to your end users. Part of the software validation process is specifying how the tool is used; this can take the form of modeling guidelines (MAAB Style guide), defined test frame works or other workflow tools.

The only time is NOW

As these tasks start mounting up you can see why the “better late than never” will not work for a safety critical workflow; by the time “late” comes along, you have already been developing algorithms without the guidelines, creating artifacts without traceability and using tools that may, or may not, be certifiable.

There is, of course, good news. Processes learned in one project can be reused (as can verification artifacts) so much of what you a face is a one time upfront cost. If this is your company’s first time, you can also leverage industry best practices such as the IEC Certification Kit.

Footnotes

1. People often talk about things being dangerous that are not, in fact, dangerous. However when it comes to safety standards, the failure to start early can result in critical mistakes entering into the project which can lead to injury and even death.
2. Hooks is a term used to describe the infrastructure to connect different components together.
3. A joke about “paper airplanes” would make sense about now.
4. Since an “outline” is a “tracing”
5. The longer you work in the safety critical world, the more often you will hear the term “safety critical”
6. And for a software development project there will be multiple “ducks” to validate from Test vector generators, code generators, compilers, test harness builders….