Tag: #Model-Based Design

Ring! Ring! Event driven architectures(1)

Published on by Michael BurkeLeave a comment

Broadly speaking, control systems are broken down into event and continuous(2) time driven domains. Continuous systems are always taking in stimulus and putting out responses. While event driven systems respond when the event happens, the rest of the time they sit patiently quite unlike an 8 year old waiting for the gates of Disneyland to open.(3) A challenge arises when you start to mix the two types of systems together; how do you mix the chocolate of continuous time with the peanut butter of events to create a greater whole?(5)

Everything was going…
smoothly then there was a break

Events interrupt the smooth execution of continuous time systems. If the event preempts the execution of the continuous time system, you are halting the logic and calculations of the system. The return to the execution of the system can pose challenges. Let’s take a look at a few…

Something’s changed

The most common problem that occurs is that when the interrupt event happens there is an overall change in the system state and the data (pre and post interrupt) is no longer in correspondence.(6) This can result in incorrect calculations and / or commands.

Falling behind

In some instances the code associated with events can take a long time; as a result the periodic update of the continuous time system can “fall behind”; potentially missing multiple updates. In some instances important data is missed during that “time out.”

What to do?

Like many situations there is no “one-size fits all” solution; but there are general rules of thumb.

  • When something has changed:
    • Buffered or protected, data can be used to resume execution of the code with contiguous data.
    • A “return from event” handler can be used to determine if a system reset is required.
  • Falling behind:
    • For the “important” events a buffer can be constructed within the event driven code.

Testing the wind

Testing interrupted code is a sticky problem as by definition, when the interrupt happens it is an unknown. Use of random interrupt generators provide one method for testing the code but cannot provide full coverage. The recommendation for this blog is to create “worst case” interrupts; e.g. look at your model and ask “what is the worst place where the execution could be interrupted” and then do it there…

Worst place for an emergency

  • In the middle of nested if/else-if/else logic: The branch you are in may no longer be valid.
  • Performing differentiation: When your “dt” changes the diff can wiff(7)
  • Fault / error detection: If the code’s job is to detect errors then, well, this is one area where you may want to protect against interrupts.(8)

Footnotes

  1. Back at the dawn of time “Ring-Ring” was the sound that a phone made, a common “event driven” experience for many people.
  2. I went back and forth 2 or 3 times on the “continuous time” driven domains. The world we live in is continuous time, it keeps on slipping into the future. However most modern control algorithms are implemented on discreet time hardware and require a discreet time implementation.
  3. Like an 8 year old who can happily read a book while waiting for the gates(4) to open, the poor control algorithm just sits there waiting for the event to ring.
  4. That comment may or may not be autobiographical.
  5. I want to distinguish between states end events; technically the shift from Neutral to First gear in a car is an event; however, it operates within the set of data for the continuous controller. The types of events to be examined are often called “asynchronous” events.
  6. Back in the old days control algorithms would send long letters betwixt each other, now with email, texts and phones correspondents are less common
  7. Wiff: to miss, to have an unpleasant smell. Note: in general, integration operations are interrupt tolerant as they average values over time.
  8. Events and errors come in all levels of severity; setting the priority of the event and the given error handling code enables your OS to arbitrate between the two functions.

Let the knife to do the work…

Published on by Michael BurkeLeave a comment

When you read about how to cook you will often read the phrase “when cutting, let the knife do the work.” There is, however, a single word missing that changes everything; it should read let the sharp knife do the work.” One word, big difference.

The difference “a” word makes

The right tool, set up correctly

During this pandemic I have finally learned how to sharpen knives(1) by hand for when I cook something yummy for my wife and I; let’s talk about how to “sharpen your models.”(2)

Model configuration: every model in Simulink has a set of parameters that defines how it executes and how code is generated.(3) Fortunately there is a simple utility that allows you to configure your model parameters for your target behavior.

You must specify basic behavior such as time domain, step size, and the target hardware. But once that is done, the balance of your configuration is handled through the specifications of your objectives.

Data settings: from your parameters to your signals, configuring your data creates more efficient and more controlled generated code. Consider setting the data type, storage class, and possibly units.

Block selection: mixing model domains such as discrete and continuous time blocks can result in sub-optimal performance. If multiple domains are required, partition the domains off using atomic boundaries

Final thoughts

These are of course just a slice(4) of the type of configuration you should consider when setting up your model. Ideally there is a group that is responsible for the basic set up and maintenance of your working environment. This should extend to all of the tools in your development process. For more in depth information on these tools, take a look at the CI, and version control posts in this blog.

Footnotes

  1. Please note, this is not a product endorsement, it is a process endorsement. Also, we don’t have the “stropping block” but I always thought it looks more fun to have the corded strop.
  2. Note, this isn’t a perfect analogy as when you sharpen your configuration you are done for the project, unlike knives that need regular re-sharpening.
  3. When we were first developing this tool I asked the question: How many configuration parameters does a Simulink model have, 113, 178, 238, 312? The correct answer was all of the above depending on the baseline configuration settings. As you can see, this tool is very useful.
  4. Okay, I couldn’t resist one last knife reference for the post.

Reuse is a State(flow) of mind…

Published on by Michael BurkeLeave a comment

I’ve written multiple posts on when and why you should reuse software components. Today I want to look at the specific use case of reusable Stateflow Atomic States.

Simple example

The most common example of reusable states I have worked with involves fault detection and management. In this simple example we start off with “NineOclock” and “all = well.” Based on an input “errorCon” we can move to “all = soSo” and finally, if we don’t recover, “all = endsWell.” In this example the transition conditions are inputs to the system (e.g., we are calculating the value of “errorCon” and “recCon” outside of the system. This works for simple examples, but what if the condition logic is more complex? What should we do then?

Parametrized Transitions!

The answer is the use of parameterized function calls. In this example the function “errorCheck” takes 3 arguments, inst, arg1, and arg2. The argument “inst” controls which branch of the switch/case function is used to calculate the transition logic. (Note: you do not need to use all of the input arguments for any given case, however they all must be passed in).

Reuse is still reuse…

Reuse with Stateflow charts has the same limitation of any other reusable function; e.g., the data types for the inputs / parameters need to be consistent across each instance of the chart. Additionally if you want to use the function based method for transition, the MATLAB functions need to be set as globally visible.

Finally, while global data can be used between instances of the chart, since it can be written to in multiple locations, care should be taken that the information is not overwritten. The most common solution to this issue is to use “bitand” functions to turn on or off bits.

Critical conditionals: initialization, termination and other conditional functions

Published on by Michael BurkeLeave a comment

Like many people, the COVID lock down has given me time to practice skills; I have been spending time practicing my (written) German; so if you skip to the end you can see this post “auf Deutsch”

Back to the start

Stoplights provide us with information, Green = Go (Initialize), Red = Stop (Terminate) and Yellow, according to the movie Starman, means go very fast. A long term question, within the Simulink environment has been, “what is the best way to perform initialization and termination operations?”

Old School: Direct Calls in C

Within the Simulink pallet, the “Custom Code” blocks allow you to directly insert code for the Init and Termination functions. The code will show up exactly as typed in the block. The downside of this method is that the code does not run in Simulation. (Note: This can also be done using direct calls to external C code. In these cases, getting the function to call exactly when you want can be difficult)

State School: Use of a Stateflow Diagram

A Stateflow Chart can be used to define modes of operation; in this case, the mode of operation is switched either using a flag or an event. This approach allows you to call any code (through external function calls or direct functions) and allows for reset and other event driven modes of operation. The downside to this method is that you need to ensure that the State flow chart is the first block called within your model (this can be done by having a function caller explicitly call it first).

New School: A very economical way

The “Initialization,” “Termination” (and “Reset“) subsystems are the final recommended methods for performing these functions. The code for the Initialization and Termination variants will show up in the Init and Term section of the generated code. Reset functions will show up in unique functions based off of the reset event name.

Within this subsystem you can make direct calls to C code, invoke Simulink or MATLAB functions and directly overwrite the state values for multiple blocks.

Best practices for Init and Term

MATLAB and Simulink have default initialization and termination functions for the model and the generated code. The defaults should only be overridden when the default behavior is incorrect for your model. There are 4 common reasons why a custom Init / Term functions are required; if you don’t fit into one of these, determine if you should be using this.

  • Startup / Shutdown physical hardware: for embedded systems with direct connections to embedded hardware, the Init / Term functions are required. (Note: it is a best practice to try to have your hardware systems in models external to the control algorithms. This allows you to “re-target” your control algorithm to different boards easily
  • One time computations: Many systems have processor intensive computations that need to be performed prior to the execution of the “body” of the model.
  • External data: as part of the startup / shut down process, data may need to be saved to memory / drive.
  • You just read: a blog and you want to try things out… I’m glad you want to try it, but review the preceding 3 reasons.

Bonus content

As promised, the results of practicing my (written) German skills. Und so

Ampeln liefern uns Informationen: Grün = Los (Initialisieren), Rot = Halt (Beendigungsvorgänge) und Gelb bedeuten laut Film Starman, sehr schnell zu fahren. Eine langfristige Frage in der Simulink-Umgebung lautete: “Was ist am besten, um Initialisierungs- und Beendigungsvorgänge durchzuführen?”.

Old School: Direct Calls in C


Innerhalb der Simulink -Palette können Sie mit den Blöcken “Benutzerdefinierter Code” direkt Code für die Funktionen Init und Beendigungsvorgänge existiert. Der Code wird genau so angezeigt, wie er im Block eingegeben wurde. Das Problem bei dieser Methode ist, dass der Code in Simulation nicht ausgeführt wird. (Eine dinge: Dies kann auch durch direkte Aufrufe von externem C-Code erfolgen. In diesen Fällen kann es schwierig sein, die Funktion genau dann aufzurufen, wenn Sie möchten)

State School: Use of a Stateflow Diagram

Ein Stateflow Chart kann verwendet werden, um Betriebsmodi zu definieren. In diesem Fall wird der Betriebsmodus entweder mithilfe eines Flags oder eines Ereignisses umgeschaltet. Mit diesem Ansatz können Sie einen beliebigen Code aufrufen und zurücksetzen und andere ereignisgesteuerte Betriebsmodi ausführen. Die Einschränkung diesmal Methode ist, dass Sie sicherstellen müssen, dass das Stateflow-Diagramm der erste Block ist, mit dem in Ihrem Modell aufgerufen wird (dies kann erfolgen, indem ein Funktionsaufrufer es explizit zuerst aufruft).

New School: A very economical way


Die Modellblöcke “Initialisieren”, “Beendigungsvorgänge” (und “Zurücksetzen”) sind die endgültige Methode zur Ausführung dieser Funktionen. Der Code für die Initialisierungs- und Beendigungsvorgänge Optionen wird im Abschnitt “Init” und “Term” des generierten Codes angezeigt.
Rücksetzfunktionen werden in eindeutigen Funktionen angezeigt, die auf dem Namen des Rücksetzereignisses basieren.

Innerhalb dieses Subsystems können Sie C-Code direkt aufrufen, Simulink- oder MATLAB-Funktionen aufrufen und die Zustandsraum Werte für mehrere Blöcke direkt ersetzen.

Best practices for Init and Term

MATLAB und Simulink verfügen über standardmäßige Initialisierungs- und Beendigungsfunktionen für das Modell und den generierten Code. Es gibt vier häufige Gründe, warum benutzerdefinierte Init / Term-Funktionen erforderlich sind.

Preview(opens in a new tab)

  • Physische Hardware starten / herunterfahren: Für eingebettete Systeme mit direkten Verbindungen zu eingebetteter Hardware sind die Init / Term-Funktionen erforderlich (Eine dinge: diese beste Vorgehensweise ist Ihre Hardwaresysteme außerhalb der Steuerungssysteme zu haben. Auf diese Weise konnen Sie Ihre Software schnell nue Hardware portieren.
  • Einmalige Berechnungen:Berechnungen erforderlich vor dem Start des Steueralgorithmus .
  • Externe Daten: Daten, die in externe Quellen geschrieben werden.

I think I’ve written this before: Revisiting Reuse

Published on by Michael Burke2 Comments

I’ve seen the question many times “Why do you care so much about reuse?” So giving my reusable answer I say “when you do something from scratch you have a new chance to make the same mistakes.” (1) If you look at your daily work, you will see we already reuse more than we realize.

When I go looking for images for “reuse” what I find most of the time are clever projects where you take a used plastic bottle and make it into a planter, or an egg carton becomes a place to start seeds.(2) What I want to talk about today is reuse for the same purpose, e.g. reuse like a hammer, a tool that you use to pound and shape the environment over and over.(3)

Hammer time(4)

Why do I care about reuse? Reuse is a company’s greatest asset; it is the accumulated knowledge over time. No one talks about “reusing a wheel” but that is what we are doing, we are reusing a highly successful concept.

So how do we get into the wheel house?(5) The first step is to identify a need, something that you (or ideally many people) need to use / do regularly.

When writing tests I frequently need to get my model to a given “state” before the test can begin. Creating the test vectors to do that manually is time consuming and error prone.

Once you have done that, think if there is a way that the task can be automated.

The solution I found was to leverage an existing tool, Simulink Design Verifier, and use the “objective” blocks to define my starting state. The tool then finds my initial test vectors to get me to where I want to be.

As described right now, this is a “proto-wheel.” It is a design process that I use (and have taught customers to use) but it is not fully reusable (yet). Why is that?(6)

Horseshoes and hand grenades(7)

This fails the “wheel test” in 2 fundemental ways

  1. It isn’t universal: every time I use it I need to recreate the interface, e.g. manually define the goal state
  2. It may not work: there are some conditions for which this approach will not find a solution.

Becoming a wheelwright: doing it right

If you want this to become “wheel like” you need to address the ways in which it fails. Here is how I plan to do that.

Create a specification language: by creating a specification language I, and my customers, will be able to quickly define the target state. Further, the specification language will ensure that errors in specification do not enter into the design

Analyze the design space: when a tool doesn’t work there is a reason; in some cases it can be deduced through mathematical analysis, in others, through analysis of failure cases. I am currently “tuning the spokes” on this wheel.

But will it roll? What is my (your) role (8) in making it happen?

In the end, a good idea and strong execution is not enough. The key to widespread reuse is getting it used by people outside the original (in this case testing) community. Until you do that it is only a specialty hammer.(9)

Getting those outside people to adopt a new tool or method is about getting people to care about the problem and the solution.(11)

Picking up a “hammer” is a 4 step process

  1. Know you have a problem: sometimes when you have been done something one way for a long time you don’t even realize there is an issue.
  2. Know there is a solution: if you don’t know hammers exist you will keep hitting things with rocks. It gets things done but your hand hurts.
  3. Have time to try out the solution: even the best hammer can be slower than your trusty rock the first few times you use it.
  4. Give the hammer maker time to make you a better hammer: chances are even the best tool will need refinement after the first few users.

Final comments: Why now?

Reuse reduces the introduction of errors into the system. Remember, “when you do something from scratch you have a new chance to make a the same mistake.”(13) And when working remotely during Covid, the chance to do so increases. Start looking at the tasks you do regularly and ask the need and automation questions.

Footnotes

  1. My wife Deborah always raves about my strawberry rhubarb crisps. But even after making over 100 of them over our 25 years together, I still can get the sugar to corn starch to lemon ratios a bit wrong if I don’t watch what I am doing.
  2. From a total energy usage I do question if we would be better off just recycling the bottle and egg carton.
  3. I started to think of the song “If I had a Hammer.” As it is a Pete Seeger song, it isn’t surprising that it is about civil and social rights. As a kid, the line “I’d hammer out love” seemed odd to me; hammers were blunt tools. When I got older I saw the other uses of hammers, to bend and shape things, to knock things into place. When you write software, write it like it’s a tool that can do all the things you need it to do.
  4. As a child of the 80’s “U Can’t Touch This” (hammer time) was at one time on nearly constant replay on the radio.
  5. First, make sure you are developing in an area that you know well so you know what needs to be done over and over.
  6. When I started this blog, I referenced the serial novels of the 19th Century. There are times I am tempted to end a blog post on a cliff hanger, but not today.
  7. When I first wrote the section title I thought “this must be a modern phrase” as while horse shoes have a long history, hand grenades are relatively new (e.g. 200 year old?) I was wrong. The earliest version can be traced back to Greek fire (or earlier)
  8. Homonyms, as a lover of puns, have always been something I have loved.
  9. If you thought I was done with the hammer metaphor you were wrong, I’m bringing it back at the end to drive my point home(11)
  10. Because that is what you do with hammers
  11. A blog post about this methodology would be one way of getting people to know about it.
  12. And in the spirit of reuse, I reused this from the start

Testing your testing infrustructure

Published on by Michael BurkeLeave a comment

Ah tests! Those silent protectors of developments integrity, always watching over us on the great continuous integration (CI) system in the clouds. Praise be to them and the eternal vigilance they provide; except… What happens to your test case if your test infrastructure is incorrect?

Quis custodiet ipsos custodes(1)

There are 4 ways in which testing infrastructure can fail, from best to worst

  1. Crashing: This is the best way your test infrastructure can fail. If this happens the test ends and you know it didn’t work.
  2. False failure: In this case, the developer will be sent a message saying “fix X”. The developer will look into it and say “your infrastructure is broken.”(2)
  3. Hanging: In this case the test never completes; eventually this will be flagged and you will get to the root of the problem
  4. False pass: This is the bane of testing. The test passes so it is never checked out.

False passes

Prevention of false passes should be a primary objective in the creation of testing infrastructure; the question is “how do you do that?”

Design reviews are a critical part of preventing false passes. Remember, your testing infrastructure is one of the most heavily reused components you will ever create.

While not preventing false positives, adherence to standards and guidelines in the creation of test infrastructure will reduce common known problems and make it easier to review the object

There are 3 primary types of “self test”

  1. Golden data: the most common type of self test is to pass known data that either passes or fails the test. This shows if it is behaving as expected but can miss edge cases(3)
  2. Coverage testing: Use another tool to generate coverage tests. If this is done, then for each test vector provided by the tool provide the correct “pass or fail” result.
  3. Stress and concurrency testing: For software running in the cloud, verification that the fact that it is running in the cloud does not cause errors(4)
  4. Time: Please, don’t let this be the way you catch things… Eventually because other things fail, false positives are found through root cause analysis.

Final thoughts

In the same way that nobody(5) notices water works until they fail, it is common to ignore testing infrastructure. Having a dedicated team in support is critical to having a smooth development process.

Footnotes

  1. In this I think we all need to take a note from Sir Samuel Vimes and watch ourselves.
  2. There is an issue here; frequently developers will blame the infrastructure before checking out what they did. Over time the infrastructure developers “tune out” the development engineers.
  3. Sometimes the “edge cases” that golden data tests miss are mainstream but since they were not reported in the test specification document, they are overlooked by the infrastructure developers.
  4. The type of errors seen here are normally multiple data read / writes to the same variable or licensing issues with tools in use.
  5. And if you look at it with only one eye, failures will slip passed.

This is “only” a test

Published on by Michael BurkeLeave a comment

In the last blog I introduced the best practices for designing scenario based tests. Today I am going to cover the, non Herculean(1), task of generating test vectors.

Good vector definitions have resolution down to the smallest time step

The “giddy” set-UP

Starting off happily let’s consider 3 things; the unit under test, the test harness and the analysis method.

  • Unit Under Test (UUT): The UUT is what you are testing. For the test to be valid, the unit must be fully encapsulated by the test harness. E.g. all inputs and outputs to the UUT come through the test harness.(2)
  • The test harness: (3)Provides the interface to the UUT, providing the inputs reading/logging the outputs. Test harnesses can be black, white or grey box. Test harnesses can be dynamic or static.(4)
  • Analysis method: Dynamic or static; how the results of the test execution are evaluated.

Not to put the cart before the horse but; we start with a test scenario. We need the test vectors. To have test vectors, we need a test harness. To have a test harness we need a well defined interface.(5)

Within the software testing domain (which includes MBD) a well defined interface means the following:

  • All the inputs and outputs of the system are known: Normally this is through a function interface (in C) or the root level inputs / outputs in a model
  • Type and timing are known: The execution rate (or trigger) for the UUT is known as are all of the data types and dimensions of the I/O.

Time to saddle up!

No more horsing around, once you have your interface designed, it is time to create your test harness. Given that we are working in the domain of Model-Based Design, the ideal objective is to automatically generate a test harness. (To all the neigh sayers out there)

A well defined interface!

Signal time!

There are four basic methods for creating signals

  • Manually: Ah…good old fashioned hand crafted test vectors. These take the most time but is where we normally start.
  • Automatically (general constraint): The next step up is to create test vectors using an auto generation tool. These tools generally allow for basic “types of tests” to be specified such as range, dead code, MCDC.
  • Automatically (constraints specified): The final approach is to use a test vector generation tool and apply constraints to the test vectors.
  • From device: Perhaps this is cheating, but a good percentage of input test vectors come from real world test data. They have all the pros and cons(6); noise and random data; they may not get what you are looking for but…

UUT and constraints

In this example we have the UUT and a “Test Assessment Block” as our method for imposing constraints. What we program into the Assessment Block is what we want to happen, not what we are checking against(7). For example, we could specify the input vectors for the WheelSpeed, WheelTqCmd and SlipRationDetected are at a given value and that the output vector is ABS_PWM . The automatic test vector generation would then create a set of tests that met that condition. You could then check for the cases where the ABS_Fault should be active.

COVID-19 Acceleration: issues with “from the device”

When you social distance from your co-workers you are, more often then not, social distancing from your physical hardware. This directly impacts the ability to gather “real world” test data. My prediction is that we will see 4 trends as a result.

  1. Greater use of existing real world data / public domain data sets: Lets be honest, there are times that data is gathered because it is easy to do so; go to the lab run the widget, collect the data and go. However there is, no doubt within your company and within government, and university data bases a wealth of existing data that will match what you need down to the 90% level
  2. Increased automation of test data collection: To some extent being in a lab or in a vehicle will always be required for collecting data, however many of the processes around setup, data collection and data transmission can be automated to reduce both the time on site and the frequency of the time on site.
  3. Improved physical models: I know what you are thinking, this is about collecting real world data! What sort of trick is this(8)! What I am suggesting is that collection of physical data will be prioritized for the creation of better physical models to reduce the net time in lab.
  4. In use collection: The next step will be the transmission of data from existing objects in the field back to the manufacture. The model “IC-2021” freezer in the field will, most likely, share 95% of the same hardware and software. This means you have a lab in the field.
The Lambert projection for more projects see

All of these methods will be used going forward to supplement traditional real-world data collection methods. With the physical modeling approach I am going to dive into how to select data to collect to rapidly improve the models. With the “in the field” we will look take our first look at big data methods.

Final thoughts

Test vectors are just one part of the overall testing infrastructure; the necessary starting point. We are going to keep looking at all the points along the Verification and Validation process; both in depth and at the impact that COVID conditions continue to have.

Footnotes

  1. With the use of one last Greek hero of antiquity, I hope to build a metaphor for the 12 labors of Hercules as applied to testing (with far fewer labors)
  2. We will look at how large the UUT should be in another blog post. For now, we will give the ballpark that a UUT should be linked to 5 ~ 8 related requirements. Each requirement will have multiple tests associated with it.
  3. A good test harness should be like the harness for a horse, e.g. provides a secure connection to the horse (software) enabling it to run fully, have the minimum number of attachment points (e.g. don’t overload with test points) and connect without chaffing (crashing or changing the behavior of the code).
  4. A dynamic test harness has the test validation software as part of the test harness, e.g. the UUT is evaluated as the test is run. A static test harness simply logs the data for post processing.
  5. Step 1 is to swallow a fly, today you will learn why!
  6. Noise is, and is not a problem. Since it will exist in the real world you should welcome noise into your test cases since that is what you will find once you deploy your product once and for all.
  7. As an example of what we want to happen, we may want to get an dessert (objective) but do not want one with coconut flavor (test).
  8. Not a very good trick, and 8! is 40,320.

Best practices for model cleanup

Published on by Michael BurkeLeave a comment

In this blog I have written a lot about “mushroom” and “spaghetti” code; today I’m going to write about the best practices for updating and cleaning up those models

Should I update?

Before you start you should ask yourself three questions

  1. Beyond cleanup are there additional modifications needed to the model? (No)
  2. Is the model, as written, performing it’s intended function? (Yes)
  3. Do I have tests cases that cover the full range of behavior of the model? (Yes)

If you answered as indicated (no,yes,yes) then stop. Spend time on another part of your code that does not meet those criteria(1). Otherwise lets start…

Baselining the model

The first step in cleaning up code is baselining the model. This activity consists of N steps

  1. Back up the model’s current state: Ideally this is already handled by your version control software but….
  2. Generate baseline test vectors: To the degree possible create baseline tests, these could be auto-generated.
  3. Generate baseline metrics: Generate the baseline metrics for the model, ram / rom usage, execution time, model coverage…
  4. Create the “Difference Harness”: The difference harness compares the original model to the update model by passing in the initial test vectors and comparing the outputs.

What is different about today?

The next question to ask in your refactoring is “do I re-factor or do I redo”? Depending on the state of the model there are times when simply re-doing the model from scratch is the better choice. This is often the case when the model was created before requirements existed and, as a result, does not meet them; that would make for a very short article though so let us assume that you are refactoring. First figure out what needs and what should change. To do that ask the following questions.

  • Review the requirements: what parts of the requirements are met, which are incorrect and which are missing?
    • Prioritize missing and incorrect requirements
  • Is it possible to decompose the model into sub-components: In most cases, the answer is no, or yes but it is tangled. It wouldn’t be mushroom code if you could.
    • Create partitioning to enable step-based modifications
  • Identify global data and complex routing: Minimization of global data should be an objective of update, complex routing is an indication that the model is not “conceptually” well decomposed
    • Move sections of the model to minimize signal routing and use of global data
  • Identify the “problem” portions of the model: Which sections of the model most frequently has bugs?
    • Squash them.

Once you have asked these questions you understand your priorities in updating the model

Begin modification

First understand the intent of the section of the model, either through inspection or through review of the requirements . Once you understand what the intention is you can start to simplify and clarify.

  • Simplifying logical statements / state charts
    • Run tool such as Simulink Design Verifier to check for dead branches, trim or fix
    • Look for redundant logical checks (multiple transitions all using the same “root” condition check)
    • Look for redundant states (multiple states exist all with the same entry and exit conditions)
  • Mathematical equations
    • Did they create blocks to replicate built in blocks? (Tables, sine, transfer functions)
      • Replace them with built-in blocks
    • Are complex equations being modeled as Simulink blocks?
      • Replace them with a MATLAB function
  • Size (to big or to small)
  • Partitioning rationale

Footnotes

  1. With mushroom code it is highly unlikely that you have tests cases that cover the full range of behavior of the model; model (or code) coverage should not be confused with full behavioral coverage since it is possible to auto-generate tests cases that would cover the full model without every understanding what that coverage means
  2. One advantage of having this blog for 3+ years is I can mine back article for information. Hopefully you will as well. What I mine is yours, nuggets of MBD wisdom.

Interface control documents and data dictionaries

Published on by Michael Burke1 Comment

Interface control documents (ICD) and data dictionaries are two parts of a mature MBD infrastructure. The question I often hear is “what is the boundary between the two artifacts”? First a high-level refresher:

  • The Data Dictionary: an artifact used to share a set of common data definitions external to the model and codebase.
    • Objective: provide common and consistent data definition between developers
  • The ICD: an artifact used to share interface information between components external to the model and codebase; often derived from or part of the requirements document set.
    • Objective: provide a common interface definition to simplify the integration of components when multiple people are working on a project.

An example of an ICD spec is

function namemyIncredibleFunction
function prototype(double mintGum, single *thought, something *else)
Call rateevent-driven
Multi-thread interruptibleyes
Function information
VariableTypeDimensionpassby
mintGumdouble1value
thoughtsingle4reference
somethingstructure10reference
function specification

And here is where the boundary question comes up. In specifying the data type and dimension in the ICD I am duplicating information that exists in the data dictionary; violating the single source of truth objective.

Duplication can be dangerous

So what is the flow of information here? I would suggest something like this…

  • The ICD document is created as part of the initial requirement specifications
  • The data interface request is used to inform the initial creation of data in the data dictionary
  • Once created the data is owned by the data dictionary

Infrastructure: making your artifacts work for you

Data dictionaries serve an obvious purpose, they are a repository for your data. On the other hand, interface control documents can seem like a burdensome overhead; which it will be without proper supporting infrastructure. If you remember the objective of the ICD, to simplify integration, then the need for tool support becomes obvious. When a developer checks in a new component it should be

  • Checked against its own ICD
  • Checked against the ICD for functions it calls and is called by
  • Its ICD should be checked against the data dictionary to validate the interface definition

With those three checks in place, early detection of invalid interfaces will be detected and integration issues can easily be avoided.

ICDs and the MATLAB / Simulink environment

Recently MathWorks released the System Composer tool. While I have not had a chance to try it out yet it offers some of the functionality desired above. I would be interested to learn of anyone’s experience with the tool

Modular testing environments

Published on by Michael Burke2 Comments

Foundations define and limit the structures we create; this is as true in Model-Based Design as it is in architecture.  With that in mind, I want to use this post to discuss the concept of modular testing environments (MTE).  First, I will point to an earlier blog post “Testing is software“, before I drill deeper into the concept of MTE.

What is a modular testing environment?

A modular testing environment consists of 5 parts

  1. Test manager: a test manager provides the framework for running, evaluating and reporting on one or more test cases. Further, the test manager provides a single hook for the automation process.
  2. Test harnesses: a test harness is the software construct that “wraps” the unit under test.  Ideally, the test harness does not change the unit under test in any fashion; e.g. it allows ‘black box’ testing.
  3. Evaluation primitives: the evaluation primitives are a set of routines that are commonly used to evaluate the results of the test.  Evaluation primitives range from a simple comparison against an expected value to complex evaluations of a sequence of events.
  4. Reporting: there are two types of reports, human and machine readable.  The human readable reports are used as part of the qualification and review process.  Machine-readable reports are used for tracking of data across the project development.
  5. Data management: testing requires multiple types of data, inputs, outputs, parameters and expected results.

Why is a modular testing environment important?

Having helped hundreds of customers develop testing environments the 5 most common issues that I have encountered are

  1. Reinventing the wheel, wrong:  Even the simplest evaluation primitive can have unexpected complexities.  When people rewrite the same evaluation multiple times mistakes are bound to occur.
  2. Tell me what happened:  When tests are pulled together in an individual fashion it is common for there to be limited or inconsistent reporting methods.
  3. Fragile tests: A fragile test is one where if the inputs change in a significant fashion the test has to be completely rewritten.
  4. “Bob” has left the company:  Often tests are written by an individual and when that person leaves the information required to maintain those tests leaves with them.
  5. It takes too much time:  When engineers have to build up tests from scratch, versus assembling from components, it does take more time to create a test.  Hence, tests are not written.

Final thoughts

Verification and validation activities are central to any software development project, Model-Based Design or otherwise.  The easier you make the system to use the more your developers will embrace them.