Soccer or Football? Multiple names for the same thing…

For those of you reading in the distant future, e.g. more than a month from now, let me set the stage.  It is July 2018 and (as an American) World Cup Soccer is in full swing.  Now if I was anyplace else in the world it would just be “The World Cup”.  However, with either name you know what I am talking about; this is what is called a “one-to-one” mapping.

In natural, e.g. spoken languages, these “one-to-one” mappings (or near mappings) are common and can often be understood through the context of the conversation.  However, in software these mappings are problematic.

A rose by any other name… may still have thorns

Multiple51C01-RUL6L._SX355_ names come into existence for multiple reasons.

  1. Multiple developers: This is the most common reason, multiple developers working in separate models/files.  They each “need” to use a variable so they create a name for the variable.
  2. Units: For physical qualities, such as vehicle speed, it is common to see, VehSpeedMPH and VehSpeedKPH.  While this may seem like a good idea, e.g. the units are known, this is perhaps the most problematic duplicate as you will often see the two instance out of sync with each other.
  3. Reusable functions: In this instance, the same function is used in multiple instances. In this instance, the key observation is to have meaningful generic names in the function.
  4. Common usage: For a subset of cases the reuse of names should be encouraged.  This is the scoped data with common usage.  For example “cnt” can be used for a counter in multiple models/functions.

The reconciliation project

First,Image result for reconciliation determine if it is worth doing.  While consistency is important, code that is fully developed and validated may not warrant the work required to update the code.  Once the decision has been made to update the code/model base the following steps should be taken.

  1. Identification:  find where the same data concept is used with different names.
  2. Select a common name: Use of a descriptive name is important in these cases to smooth the transition process.
  3. Check for “unit” issues:  If the different names existed due to desired units, validate that the change to the common units is accounted for downstream.
  4. Update documentation/test cases: (You do have those right?). Documentation and test cases will often, reference the “old” names.  Update the documentation to reflect the “new” name.
  5. Validate the units: After the updates have been performed the full regression test suite should be run to validate the behavior of the models.

 

 

Is it a guideline if you can’t enforce it?

As the past coordinator for the MAAB Style Guidelines, I have spent a fruitful number of hours thinking about guidelines for the Model-Based Design and Safety Critical environments. In a recent discussion, I was challenged by the question “Why have a guideline that you cannot check”?

Now any guideline can be validated through a manual review process.  In this instance, the query was specifically asking about automatic validation.  Further, they were working in an environment where the majority of users were new to the Model-Based Design environment.  So here is my, evolved, answer.

Why can’t it be enforced?

Some guidelines cannot be enforced because they depend on human judgment, things like “meaningful names” or “readable diagrams” are, by their very nature subjective.  (Though I have an idea for a neural network solution for the meaningful names issue).  Since that can’t be enforced is it worth throwing out?  Generally no; it becomes a “best practice” and perhaps a subset of the rule could be enforced (e.g. limit the number of blocks per level of the model, minimum name lengths…)

Image result for enforcer

What do you do with the non-enforceable?

As a general best practice when guidelines are rolled out there should be an education seminar to explain the guidelines and their purpose.  Special emphasis needs to be placed on those guidelines that cannot be automatically enforced.  Explain the

  • Rationale: why the guideline benefits the end user
  • Effort: how hard will it be for the user to follow
  • Benefit: what does the end user and the team get out of following the guideline

Final thoughts

In the end, these guidelines should be thought of as a recommendation.  Some, but not all will be caught during reviews with co-workers and by test engineers.  That they will not always be followed should be expected but if you never provide the guidance they never can be followed.  If you keep them to a minimum, say no more then 6 to 10, these guidelines that are highly impactful, well,  eventually people will follow them without thinking.

Image result for recommendations

When is the glue too thick?

The term “glue code” is a colloquial term for “a thin layer of software that connects software”.  In general, it is used to connect to software components that were not designed to interact with each other.   This is a common problem and, when sensibly done, is a fine approach.  However, it is possible to develop “super glue” solutions which are, in the end, fragile and difficult to maintain.

“Standard glue”

TheImage result for standard glue following are examples of standard “glue code” functions.   Correctly implemented they are a thin layer between modules

  • Data format translations: repackaging data between two different formats, such as an XML to CSV translation function.
  • Communication port: adding a data transmission function between two pieces of code.
  • Registry/read functions: these are functions that map data from one source (such as requirements) onto an object (such as a model)
  • Error catching: these functions, generally, work in concert with other glue code ensuring that the data exchanged between the modules is correctly formatted.

Crazy glue

How Image result for crazy glue helmetdo I tell if I have crazy glue?  There are 4 basic warning signs

  1. Use of multiple languages:  connecting two software components written in different languages is a common task.  However, if you find your glue code uses more than one language chances are you doing something to convoluted.
  2. Use of files for the interface: ideally glue code is written by leveraging existing APIs in the software components.  If the function interface is through a file and a “poling function” the data exchange will be difficult to maintain.
  3. The growth of “special cases”:  when the number of special cases the glue code has to handle gets above 10 to 15 chances are the data exchange format is not well defined.
  4. Size: there is no hard and fast rule for how large glue code can be, however at some point in time it stops being glue and becomes its own software component.

What to do with crazy glue?

In the ideal world, the glue code would be replaced with improved APIs in the source software components.  Often this is not possible due to ownership issues.  When this is the case basic software best practice come into play

  1. Break the glue into components
  2. Look for ways for one component to encapsulate the other
  3. Refactor the code to remove special cases
  4. Consider using different software components

Image result for duct tape

 

 

 

Hierachy of reuse

I have written about reuse in a number of post before; today I wanted to talk about a conceptual framework of reusability.  In this video I talk about four types of reusable “objects”

  • Concepts: These have a high reusability with a corisponding high initial development cost.  Examples include:
    • Model Architectures, testing infrustructure, modeling guidelines
  • Fundemental objects: These also have a high reusability, but have a realitivly low initial cost.  Examples include:
    • Transfer function implimentations, bang-bang control algorithms…
  • System implimentations: These have a medium level of reuse with and a moderate level of initial implimentation cost.  The “smaller” the system is the lower the cost.  Examples include:
    • An internal combustion engine model, a wing’s control surface
  • Targeted implimentation: These have the lowest level of reusability and implimentation cost.  Examples include
    • Hardware device drivers, specific fault montering algorithms.

 

 

Why testing fails…

ForImage result for improper complex systems, the goal of “complete” testing can be impossible to reach.  This often boils down to issues of time and complexity.  This is the most “common” type of failure.  A second, more insidious type of failure is improper testing.

What is improper testing?

Improper testing arises from not fully understanding what is being tested or how the testing algorithms work.

Let’s take a simple example, response time to a step input.  For this example, we will use the following test requirementImage result for step input response

TR_1028: Within 0.5 seconds of the unit step input the control system will settle to within 5% of the input value.

Failure type 1:  Single point pass:

There are two errors with this test:

  1. The hard comparison for the driver input value.  A correct method for evaluating this would be
    abs(driver-target) < eps(target)
  2. The pass condition is a single point in time.  What this means is that the signal could continue past the target not settling.

stepMessureError

Failure type 2: “Noise” intolerant

To fix this error the settling time needs to be defined.  To fix this a new state of “Settling” is added.   However, we have now introduced a  new error

  1. Hard failure condition: the response two the signal can include overshoot before the signal dampens down.  The current test fails if the signal falls out of the “less then 5%” range at any time.

failure_1_p2

Solution to this issues: 

The solution to this issue is to define a max settling time and to trigger a failure condition if the signal has not settled within the time.

failure_1_p3

Failure type 3: Sampling

A common failure mode, when dealing with large data relates to sampling.  Let’s take a look at an error flag example.

TR_2034: During the duration of operation the check engine flag will not be active.

It is common when logging data in a system to downsample the data; e.g. log every 10th or 100th data point. If the flag is intermitent it is possible that the flag would not be logged.

There are two solutions to this problem; first, the code could be refactored to include an “ever high” flag.  The second option would be to reconfigure the sampling rate for this specific test.

Failure type 4: Matching problems…

One Image result for matchescommon test type is baseline comparisons.  There are three types of problems that arise when comparing baseline data

  1. Failure to align/sampling:  baseline data is often taken from physical systems or prior simulations.  In this case, alignment of the data between the simulation and the test data is critical.  Likewise, if the sample rate of the baseline and the simulation are different then the alignment becomes more difficult.
  2. Poorly defined tolerances:  comparisons can be made against, absolute, relative and percentage error.  Further, the data type (integer, enum, double) have an effect on the type of comparison that should be performed
  3. Scaling/Units: The final, common, baseline data issue is a failure to correctly account for differences in units and scaling between the to sets of data.

Summary

The types of failures described here are a subset of improper testing issues.  They represent a failure to understand the full scope of the behavior of the system or a failure to understand how the collected data can be inspected.

Ask the right question: Architectural size

Recently a client posed a question I have heard a number of times, “How many States can I have in my model before there are problems”?  On the surface, this seems like an O.K. question, however, when we dig in a little we see the inherent assumptions with the question.

Size matters, after a fashion

AsImage result for size matters a basic metric, the number of states in a model is meaningless, it is akin to the question “how many lines of code before there are problems”?  If someone said they had a program with one function and 100,000 lines of code you would assume that it was problematic in its complexity.  On the other hand, if they said the program had 100 functions you would think that the model was well architected.  Going to the other extream if the function had 1,000 functions you may think that they have created architectural problems of increased complexity.

No one builds a house with a Swiss army knife

Models are tools, Victorinox Swiss Champ XAVT in red transparent - 1.6795.XAVTthey perform functions in response to inputs.  It is possible to build a single function that performs a 1,000 different functions but that is rarely the correct way to go.

Rather each model should be viewed as a Image result for toolboxspecialized tool performs a function or set of related functions.  Again this relates to the “100 or 1,000” functions for a 100,000 lines of code.  I generally consider something a “related function” if

  • Uses the same inputs:  E.g. the function does not need to import additional data
  • Is used at the same time: E.g. the information is used in the same larger problem you are trying to solve.

For example, calculating the wheel speed and wheel torque in the same ABS braking function makes sense as they use the same input data (generally a PWM encoder) and are used at the same time (to determine the brake pulse width).  However, calculating mileage in that function, which can be derived from the wheel speed, does not make sense as it is not part of the same problem you are trying to solve.

Keeping it in memory…

In Image result for you can remember 7 peoplethis instance, I am talking about the developer’s memory. Above a given size and complexity it becomes difficult for a developer to remember all the parts of a function operate.

As a general rule of thumb, I try to stick to  a “depth of 3” limit.  No subsystems or nested states more than three levels deep.  If there is a need for greater depth I look to see if there is a way to decompose the model or chart into referenced models and charts.  One note, when measuring “depth” the count stops when a referenced model or chart is encountered as these are assumed to be atomic systems developed independently from the parent.

Benefits of decomposition

The Image result for decomposingfollowing benefits are realized through the decomposition of models

  1. Simplified testing: large functions have a large number of inputs, outputs, and possible responses.  Smaller models have reduced testing criteria.
  2. Simplified requirements linking: Generally, well decomposed aligns with the requirements by not clumping disparent functionality together.
  3. Improved reusability: Smaller functions are more likely to be generic or easily customizable.
  4. Improved readability: A smaller model can more quickly be reviewed and analyzed then a larger model.

What is the correct question?

There are two questions I would ask:

  1. How do I make the model functionally correct?
  2. How do I make the model readable?

For guidelines on that topic, you can read my Stateflow Best Practices document.

 

 

The Model-Based Design Workflow…

The following is an idealized Model-Based Design workflow, from initial requirements to product release.  The workflow assumes a multi-person team with resources to support multiple roles.

It all starts with requirements…

Ideally, the process starts with foundational tools and processes in place.  These consist of

  • Modeling guidelines:  Covers model architecture, data usage, and model clarity
  • Testing guidelines: How the system will be validated
  • Requirements tracking: A system for tracking the compliance and changes to requirements
  • Bug reporting: A system for tracking issues as they are discovered; this is tightly coupled to the requirements tracking.
  • Support infrastructure: The supporting infrastructure includes tools such as version control and CI systems.
  • Project timeline: The project timeline provides the objects for the completion of the project and the resource allocation (e.g people)

wf_p1

From the initial high-level requirements derived requirements and derived tests are defined.  These documents, along with modeling guidelines and testing guidelines are used to create the initial system and component level models and their associated test cases.  These models and tests are entered into the requirements tracking system and assigned to the appropriate teams.

Project development

During the development phase. both the models and tests points are elaborated by adding in details from the requirement and testing documents.  As these features are added to the system the models are validated against the test cases.

wf_p2

As issues are uncovered, either way in which the design is lacking or bugs are discovered information is added to the bug and requirements tracking systems.  This workflow goes on in parallel for each developer/model.

 

This development goes on in parallel for both the system under development and any supporting models such as plant and environmental models.

System integration and release

As the maturity of individual components matures the system level integration and validation rigor is increased.  At this stage of development testing of the derived objects, e.g. generated code, increases in frequency and rigor.

wf_p3

Wrap up

After the release of the project, the components from the development should be examined for reusability.  Beyond components processes and guidelines should be reviewed for updates to the best practices.

wf_p4

Statistical variations in plant models

Plant models, whether based on first principals physics or regression models based on real-world data are a cornerstone of the Model-Based Design controls development process.  During the initial development of the control algorithms a “static” physical model is sufficient; however when the development moves into the diagnostic and release phase physical models that demonstrate real-world variation are required.

Variations, not noise…

In Image result for variationa previous blog, I wrote about the importance of noise in testing.   Variations are different from noise in that they are a constant offset.  For instance, my height will always be 6’3″ while my wife Deborah’s will be 5’10”.  If we design an airbag system assuming everyone was 5’10” then there could be issues when the first 6’3″ person is in the car with an accident.

Working with variations

If we continue the “body variations” example and think of all the variables associated with the body, height, weight, leg length, arm length… we will observe two things

  1. There is a correlation between some variables:  In general leg length increases as height increases, as does weight.
  2. There are outliers:  While there are general correlations between properties there are still outliers which cannot be ignored.

So given these two considerations how do we proceed?

Data at the boundaries, data in the center

Test Image result for at the boundarydata should be defined that includes both data at the boundaries and in the ‘center’ of the test space.  Data at the boundaries exercises the edge cases while the data in the center is used to validate the mainline behavior.  When considering which boundary conditions to include consider the following issues.

  1. For discreet variations:  In instances where the variations are discreet, e.g. on/off, flow/no-flow all discreet instances should be included
  2. For continuous variations: In the example of height, values at the endpoints should be selected along with a set of points within the range.  (The total number should be a function of what a nominal unit is in the range.  For instance, if we took a height range from 4’10” to 6’6″ and assumed the nominal unit of 1″ then perhaps a spacing of 6″ would be reasonable)

Variations and variations… working with multiple variations

In any real-world system, there are multiple parameters that will vary.  Selecting which combination of variations (outliers and central points) needs to be determined in a rigorous fashion.  In an upcoming post, I will cover how six sigma style approaches can be used to determine which points should be selected.

 

ICDs and collaboratve verification

Interface Control Documents (ICD) are a method for specifying the functional interface of a component or system.  Used correctly they prevent integration errors and promote formal development practices.

What is in the document?

At a minimum, the ICD consists of the following information

  • Input specification
    • Name
    • Data type
    • Dimension
    • Reference/value
  • Output specification
    • Name
    • Data type
    • Dimension
    • Reference/value
  • Global data used
  • Calling method
    • Periodic or event-driven
    • Reusable / non-reusable

The I/O and global data are generally well understood.  Specification of the calling method is required to understand how time-dependent functions such as integrators or transfer functions will behave.

Additional data may include information such as signal range, update rate, units…  All of this information should be derived from the requirement specification.  (Note: the ICD is sometimes viewed as a derived requirement document)

How is the ICD used?

The ICD provides a baseline baseLinefor the interface to the software component.  Once the ICD has been defined the initial model can be created.  This is sometimes called a “shell model.”  The shell model has all of the inputs and outputs as defined by the ICD document.  The shell model can then be integrated into the system level model (or another integration model) for a system lockdown.  This integration model provides the first level of testing of the interface.  If the interface of the shell model changes the integration model will break.

ICD_WF