Tag: #Model-Based Design

If / elseif/ else: Why didn’t you ask me that in the first place?

Published on by Michael BurkeLeave a comment

Conditional logic, such as if/else, switch/case constructs, truth tables and state machines are common to most programming languages.  In today’s blog, there are two aspects of conditional logic that I want to address in today’s post,  independence, efficiency versus clarity.

Independence

Let’s compare two simple examples, in the first we have a single variable “A”, in the second we have three variables “A”, “B” and “C”

iflogicelseifstartingpoint

If we consider the first case the order of the comparisons (A==1) and (A==2) can be changed with no impact on the functional behavior of the code.  This is because the two conditions are mutually exclusive (1).  However in the second case, without formal analysis,  we cannot say if the order affects the functional behavior.  That is to say, if both A, B, and C can be true at the same time then the order of evaluation impacts the resulting functional behavior (2).

Frequently people are not aware of this possibility and as a have difficulty in evaluating the functional behavior of their code(3).

Efficiency versus clarity

For most people, the standard gear shift in cars is well known, with the order of states PRNDL as a standard (Park,  Reverse, Neutral, Drive, Low).  Therefore you commonly see conditional logic (or state logic) that represents the PRNDL like this…

prndl

However, from an efficiency point of view, this is poorly defined.  The majority of the time the transitions you will see are from park to reverse, park to drive, drive to park or reverse to park.  This would suggest the following organization

prndl_eff.

Why is this more efficient?  Simply put it reduces the average number of comparison operations required to reach the desired end state.

Compound conditions

The previous example was fairly straight forward; there is a single variable with independent states.  It is more common to find compound conditions defining the if/then/else logic.  Let’s look at our friends, A, B and C again.

compoundif

This first example is a classic compound if statement.  In this example, the logic is fairly straight forward.

deeplogicSo which of these is more readable?  This is a difficult question to answer.  In this case, the indented format of the code makes the binary nature of the if/else-if/else conditions clear.  However, if I introduced more complex evaluations, non-binary operations, then it would perhaps a hybrid of the two formats would be preferable.

Rules of thumb

  1. Keep the number of comparisons on a single line 3 or less:  Exceeding 3 comparisons per line make validation of the coverage of the case difficult.
  2. Keep the depth of if/if/if trees to 3 or less:  Traceability back to the top of the tree decreases at depths greater than 3.
  3. Place the binary comparisons at the top level of your if/if/if trees:   Placing the binary operations at the top of the tree reduces the overall number of if/else required.

Final thoughts

Formatting and density of information are frequently “hot topics” of debate.  The rules of thumb listed above should not be considered a final verdict but rather a starting point for discussion.  As always I welcome all comments.

Footnotes

(1) Note this may not be true if you move to a quantum computer where a single bit could maintain multiple states.  Until then you are fine this statement holds true.

(2) The second screen shot has a common error; lack of documentation.  The code should be commented to state if the code can be reordered.

(3)The common scenario has a person perplexed why they do not call “callBsFunction” when B is true.  Not realizing that it is gated by the “A” is true if statement.

Testing is software

Published on by Michael Burke5 Comments

This blog is a re-posting of early work from linkedin; I will be re-posting this week while I am at the Software Design for Medical Devices Europe conference in Munich.

Enabling the adoption of Model-Based Design

Test early, test often, test against requirements and test using formal methods. This is the mantra that developers (hopefully) hear. But what does it mean in practice? How do you produce effective and maintainable tests? I will argue that the first step is to think of test development in the same light as software development. Good testing infrastructure has requirements, is built from reusable components and written in a clear fashion to facilitate extensions and debugging efforts.

Why should you care?

In my 20+ years working in software, 2/3 of it in a consultative role, the most common problem I am called in to work on is mushroom code(1). Mushroom code is the end result of unstructured development, new algorithms are added on top of existing algorithms with little understanding of what it is feeding on. The result is an organic mess that is hard to sort out. This is prevalent in algorithmic development and even more common in testing which is often done “late and under the gun”

Testing components

A fully developed testing infrastructure consists of 5 components, a manager, execution methods, harnesses, reporting methods, and evaluation methods.

1.) Evaluation methods: use the data created through the execution of the test to determine the pass / fail / percentage complete status of the test:

Example a.) A MCDC test the evaluation would determine the percentage of conditions taken

Example b.) A regression test could compare the output values of between the baseline version of the code and the current release.

2.) Reporting methods: take the data from the evaluation methods and generate both human readable and history reports. The history reports are used to track overall trends in the software development process.(2)

3.) Harness: the harness provides a method for calling the unit under test (UUT) without modifying the UUT. Note test harnesses facilitate black box testing, e.g. the internal states of the unit under test are not known. However if internal states of the UUT are outputs at the root level of the model then white box testing can be done using the unit under test.(3)

4.) Execution methods: is how the test is run. This could be the simulation of a Simulink model, the execution of a .exe file, static testing (as with Polyspace) or the Real-Time execution (4)of the code.

As the name implies there is more than one “execution method.” They should be developed as a general class that allows the same method (simulation) to be applied to multiple harnesses. Each instance of a execution method applied to a harness is considered a test case.

5.) Test manager: is were all of these components come together. The test manager

  • Holds a list of the test cases
  • Automates the loading of associated test data
  • Triggers the execution of the test
  • Triggers the evaluation of the results
  • Triggers the generate of the test report

Sadly it will not yet fetch you a cold beverage.

Notes

1.) Mushroom code and spaghetti code are similar that they develop due to a lack of planning. Spaghetti code is characterized with convoluted calling structures; mushroom code is accumulation of code on top of code.

2.) An interesting list of what should go into a report can be found here.

3.) Any model can be turned into white box testing if global data is used. However the use of global data potential introduces additional failure cases.

4.) Yes, this blog retreads the work from 6 months ago, however it is good to review these issues.

Software Design for Medical Devices Europe

Published on by Michael BurkeLeave a comment

I happy to say that next week, Feb 21st through Feb 23 I will be attending the Software Design for Medical Devices Europe conference in Munich.  I will be present a seminar on adoption of Model-Based Design and support one of my customers in his presentation on his experience with implementing MBD at his company.

Projects of interest (II): Listening for success

Published on by Michael Burke2 Comments

What is success? How do you define it, how do you measure it?  With software projects, it is easy to say when a project is complete but a complete project is not always a successful project.

So hear, at an abstract level, is my definition of a successful project.  The project…

  • solves the underlying (“big-picture”) requirements:  It is possible, even common, in the translating the initial (or user) requirements into derived requirements that the “big-picture” objectives are lost.  You see this reflected in tools and products that are functionally correct but more difficult to use or fail to provide the experience the user wants.
  • informs future work: A successful project can inform future work in two ways.  First directly, the work done on one project may be reused in subsequent projects.  The second is through acquisition of new knowledge(1) .
  • mentors junior people on the project: Every project is an opportunity for junior people to develop new skills and deeper understanding of the what they are working on(2) .

Background

17 years ago, on my first project as very green consultant,
I made the mistake of doing exactly what the green-consultantscustomer asked me to do.  Their request was to help them automate the routing of signals around a complex multi-level model.

I did what they asked, I learned a lot; efficient recursive programming, how to handle complex regular expressions, error handling for ill-defined systems.  The customer received a tool that did exactly what they asked for and they used it for the next 3 years.

So how was this project completed yet not successful?  First I didn’t step back to ask “what do they need?”  The customer thought they needed a way to route signals; in truth they needed a better model architecture.  The reason that they stopped using the tool 3 years later is that they realized this for themselves and developed a better model decomposition.

The second way in which this project failed is that it did not inform future work.  By its’ very nature it was a dead end tool; keeping them trapped  in an inefficient model architecture.  While I learned things that information was not applicable for my customer.

How to start succeeding?

Between the title of this post and my measures of success the answer should be clear.  At the start of my engagement I should have talked with and listened to my clients; that would have lead to the understanding that their architecture was in poor shape, I would have understood their underlying(3)  requirements.

student_success

Once you have the true objectives in mind make sure to review them periodically to ensure that the project has not drifted away from them.  Think about how the current project can inform future work, either directly through reuse or through education.  If it is for reuse budget the time in development to enable future reuse(4).

Footnotes

(1) There is a practical balance to be struck when learning new things on a project.  The time spent on learning new methods / tools should not slow down the progress of the project.  Ideally the knowledge would be gained organically while working on the project.

(2) Mentorship on projects is often informal; even basic act of discussing what and why design decisions have been made with junior colleagues will aid in their development.

(3) I am using “underlying” and “base” requirements to refer to the “big-picture” requirements from which all others are derived.  Given that the term for these big-picture requirements vary from field to field I hope that this will still be clear.

(4) Enabling reuse requires additional design and testing time.  A general rule of thumb is  to allocate an additional 10% ~15% of the development time.  I will write more about reuse in a future blog post.

Video Blog: Clarity in communication

Published on by Michael Burke2 Comments

Clarity of communication is key to any process; this short video covers a few thoughts on the importance of clarity in software development.

Selecting the initial project

Published on by Michael Burke1 Comment

When setting the scope of the initial project it is critical to remember the objectives of the initial phase as outlined in earlier posts.  They are:

  1. Understand how artifacts from models integrate with existing artifacts
  2. Establish baseline testing activities
  3. Implement version control for new modeling artifacts
  4. Identify initial model and data architecture

The motivation behind these objectives is to determine how the models fit into your overall process.  To do this a system of appropriate complexity needs to be selected.

Integration

Within the software community the term “spaghetti code”(1) is used to describe software that is poorly partitioned lacking in well-defined interfaces.  puzzleAs the name implies integration with spaghetti code is difficult to break apart and difficult to integrate in a “clean” fashion.

For the first project a section of code should be selected with a reasonably clean interface.  The primary thing to avoid is a section of code that is heavily dependent on global data.  For the initial system, you are almost always integrating the new component into the existing system.  You should target components with a limited number of I/O and no or limited dependence on global data.

(Note: this section has assumed that you are working on a project with existing code that must be integrated with.)

Questions to ask

Will the model need to…

  1. call existing supporting functions?
  2. access global data?
  3. have context dependent execution?

The objective is to select a model that minimizes those requirements.

V&V

The verification(2) and validation tasks(3) play a key part in determining the scope of the initial project.  A good module would include

  1. Modal systems:  State charts with event based logic; allows for verification of context based execution.
  2. Conditional execution: If/then/else logic and enabled / disabled subsystems; allows for testing of model coverage
  3. Dynamic systems: Requires closed loop simulation with stimulus; validates the response characteristics of the system.

11519783-vector-check-marks-stock-vector-check-box-tick

For each of these three items the expected outcome should be fairly well know; the objective is not to find major errors, though this sometimes happens during initial projects, but to understand how you will do these activities.

Questions to ask

Will the model…

  1. produce measurable outputs?
  2. have multiple modes of operation?
  3. have dynamic responses?

The objective is to select a model that maximizes these requirements.

Version control

While one of the objectives of the initial adoption is to understand the artifacts that will be placed under version control this objective has little impact on the setting of the scope of the system.  Version control will be covered in a later post.  For now you can read one of my earlier linked in blog posts on version control.

Model and data architecture

The selected component should enable you to validate your selected architecture as outlined in the model architecture and data sections of this blog.architecture

Questions to ask

Will the model need to have…

  1. parameterizable data calculations.
  2. a functional interface definition.
  3. conditional execution interfaces.

The objective is to select a model that maximizes these requirements.

Complexity

The final metric to consider is algorithmic complexity.  While Model-Based Design allows people to develop more complex algorithms, for the initial project an algorithm that is well understood should be selected.  The objective is to examine how models fit into your development workflow, not to validate the capabilities of the model(4).

Questions to ask

The algorithm should be

  1. an existing or extension of an existing algorithm
  2. part of the project that will first deploy the Model-Based Design process.

The objective is to select a model that maximizes these requirements.

Footnotes

(1) I prefer the term “mushroom code” as it better explains how this code comes about.  It is generally code that has grown on top of well written code but due to organic decay is now difficult to take apart.

(2) Verification is intended to check that a product meets a set of design specifications

(3) Validation is intended to ensure a product, that meets the operational needs of the user.

(4) The capabilities of the modeling environment should be assessed; however this is a separate task from developing your Model-Based Design workflow.

(5) The header image is a reference to the song from “The Sound of Music: Do-Re-Mi ” which provides the advice on where to start.

Initial Adoption: Leveraging existing processes

Published on by Michael BurkeLeave a comment

A common misconception around adopting Model-Based Design is the degree to which things are “new“.  For most software / engineering groups adopting Model-Based Design is an exercise that consists of three main activities

  1. Redefining the development workflow Model-Based Design allows you to perform multiple operations earlier in the development cycle than traditional C based development workflows.
  2. Refactoring existing processes: Model-Based Design allows automation of some development tasks and processes.
  3. Re-targeting process tools: The tools used by the existing processes will change to reflect the automation and simulation tools that Model-Based Design provides.

While Model-Based Design adds in additional processes; it has been my experience that customers have used 70% or more of their existing processes for items 1 and 2.  Further, the automation gain from MBD simplifies many of their existing processes.

Redefining the development workflow

Model-Based Design has been used in conjunction with multiple design workflows, most commonly I reference a “double V feedback” workflow.  Regardless of the development workflow that you follow the key development concept is the movement of simulation-based design and verification into the front end of development.

2017-02-02_8-27-41
Delta V’s(1)

One of the key arguments in favor of Model-Based Design is the ability to simulate models early in the development process to improve the design of the system through system analysis; or as we said back in my automotive days “models, not metal.”   

In the same way that earlybugs implementation of functional and system level design reduces development time the ability to test models early in the development process both reduces development time and improves the likelihood of detecting bugs early in the process.

Refactoring existing processes

Following the same principals from redefining the development workflow refactoring existing processes considers three things

  1. Is the existing process still required?
  2. Can the existing process be automated?
  3. Are there new processes requirements or opportunities?

Is the existing process still required

By moving to Model-Based Design there are some processes that are no longer required, due to automation within the tool.  For example in model-based design processes that include a code generation step a manual code review process are generally not required.

Can the existing process be automated

New methods for automating processes routinely are created; even without moving to a Model-Based Design workflow a yearly review of processes that could be automated is a good best practice.  MBD workflows tend to have high levels of automation due to their built-in integration with automation tools such as Jenkins (continuous integration) or Rational Doors (requirements management)(2).

Are there new processes requirements or opportunities

Transitioning to Model-Based Design workflow will introduce new process.  This happens for three primary reasons.  Firs, the movement to MBD was driven by out side standards and the standard (such as DO-178C) requires a process to be performed.  Second the transition allows for a new process that will improve the final product, such as design based simulation.  The final reason new processes are adopted is in support of the interface between existing and new project development tasks and objects.

Re-targeting tools used by processes

The tools used by the existing processes will change to reflect the  the automation and simulation tools that Model-Based Design provides.  This blog will goes into more detail in “The role of supporting tools in Model-Based Design” and “Model-Based Design Building Blocks” posts (3).

strategist-clipart-target

Footnotes

(1) In just a casual search I found over 50 graphical representations of the “software design V.”  As with all things make sure there is clarity around what you are talking.
(2) While both Doors and Jenkins are solid tools in their categories the selection of them is not an endorsement for that role.
(3) Currently these two sections are still a work in progress.  More detail will be filled in as this blog continues developing.

Model-Based Design: Projects of interest

Published on by Michael BurkeLeave a comment

Early in my career one of my mentors made the statement

“If we understand the system we can model it.
If we can model it we can make predictions.
If we can make predictions we can make improvements”

In the past 20+ years, I have not heard a better statement of the driving ethos behind Model-Based Design.

If we understand the system, we can model it:(What we need to do): when the system is understood, it can be described mathematically.  This could be a derived first principal model or a statistical model; the important thing is that the confidence in the model fidelity is understood.

If we can model, we can make predictions:(What we can do): once the model is known it can be used.  The use of the model can be in the design of a controller, predicting a rainfall or embedded within a system to allow the system to respond with better insight.

If we can make predictions, we can make improvements:(Why we do it): this last part is the heart of Model-Based Design.  Once we can make accurate predictions we can use that information to improve what we are doing.

Model and equation…

Models build on a foundation of equationsflightdynamics to provide a dynamic, time variant representation of the real-world phenomenon.  Moreover those equations are working as part of a system; you leverage models when you move into complex systems with multiple interdependent equations.  Within the Model-Based Design world, we most often think of these systems as closed loop system.  Similar examples can be seen in the social sciences, in biology and chemistry.

Understanding from a sewage treatment plant…

Coming from an aerospace background, and starting my working career out in the automotive industry the general nature of models sunk in during one of my earliest consulting engagement; helping a customer model a sewage treatment plant to determine optimal processing steps against a set of formal requirements.

  • Requirements
    • The plant may not discharge more than N% of water in untreated state
    • The plant’s physical size cannot exceed Y square miles
    • …
  •  Objectives
    • Minimize the total processing cost of sewage treatment (weight: ω)
    • Minimize the total processing time of sewage (weight: λ)
    • Maximize the production of  energy from bio-gas (weight: Φ)
    • ….
  • Variants of inputs
    • Sewage inflow base rate has +/- 15% flow rate change
    • Extreme storm conditions can increase flow rate by 50%
    • ….

The final system model included bio-chemical reactions, fluid dynamic models, statistical “flush rates” and many other domains that I have now forgotten.  The final model was not able to answer all of the questions that the engineers had, however, it did allow them to design a plant with significantly lower untreated discharge rates and lower sewage processing costs.  This was possible because of the models.  This was the project that showed me just how expensive Model-Based Design is.

Initial adoption: objectives and metrics

Published on by Michael BurkeLeave a comment

Objectives and metrics

Based on the information collected from the process adoption team the objectives for the initial adoption phase should be set.  While the specifics for any given organization will be different the following outline is a standard view.

1403204517-3-keys-measure-success-loyalty-program

  1. Technical 
    1. Complete 1 or 2 “trial” models
      1. Identify the initial model architecture
      2. Identify the initial data architecture
      3. Establish baseline analysis methods
      4. Establish baseline testing methods
    2. Understand how artifacts from models integrate with existing artifacts
    3. Implement version control for new modeling artifacts
  2. Managerial
    1. Review methods for measuring model key performance indicators (KPIs)
    2. Review resources required during initial adoption phase

The technical metrics

Completion of the trial models

In a future post we will examine how to select your trial model,  but for now let’s answer the question “what does it mean to complete a trial model? ”  This decomposes into the four tasks as outlined above.  The model and data architecture are covered in some depth in previous posts so let us talk about analysis and testing.

Within the Simulink domain, a fundamental aspect of a model is the ability to simulate the behavior of the plant or the control algorithm. analog-simulationThe simulation is used during the early stage of development to analyze the model to determine if the functional behavior of the model.  The developer performs elaborates the model until the behavior functionality matches the requirements.  This is verified through simulation.  Once the model meets the requirements the functionality can be “locked down” through the use of formal tests; again using simulation.

It is worth noting that some requirements will be met before others, they should be formally locked down under test as they are achieved.

Integration with existing artifacts

For most companies, unless they are starting from
integratesoftwarea clean sheet there will be existing software components that need to be integrated with those created by the Model-Based Design process.  There are three types of integration

  1. bringing existing software into the Model-Based Design framework
  2. Bringing Model-Based Design artifacts into the existing architecture.
  3. A combination of 1 and 2.

The topic of integration will be covered in greater detail in an upcoming post.  However, the fundamental guidelines for integration (in either direction) are the following.

  • Create software objects with well-defined interfaces (encapsulation)
  • Limit dependencies of the software objects on external objects
  • Minimize the use of “glue code”(1).

Version control of objects

Version control processes use tools to enable team-based development while maintaining multiple “release” or “branches”.  During the initial phase of the project which software objects will be placed under control and how a “release” will be defined should defined.  This initial definition will be refined as the Model-Based Design process is elaborated.  This blog will go into detail on this in a future post.  The basic questions are

  • Do you include derived objects in the version control software: .c, .h, test reports…
  • How do you arbitrate check in conflicts: How do engineers resolve conflicts in their model interfaces?
  • How do you share data / models across projects: What methodology will facilitate reuse of objects across multiple projects with minimal configuration impact?

Managerial metrics

The initial adoption phase by its nature will be slower than later phases as people are still learning new capabilities and processes.  57572027-gears-and-kpi-key-performance-ind-cator-mechanismThe primary objectives during this phase are

  1. Learn what bottlenecks with the new process.
  2. Understand existing issues uncovered by the transition
  3. Determine level of resources for the next stage

The discovery of “Objective 2’s”, existing issues, often surprises people.  The act of transitioning to a new process forces the evaluation of existing processes and, more often than not, illuminates existing shortcomings.  Extra care should be taken to ensure that the new process addresses those shortcomings.

In the next stage, the validation project, the team should expand beyond the initial “core” team. Ideally, a people from outside the initial project scope should be brought in to identify developmental pain points that did not exist in the “core group” processes.

Footnotes

(1): “Glue code” is a software object created solely for the connection of two other software objects.

Fidelity of plant models in closed loop simulation

Published on by Michael Burke11 Comments

In graduate school I studied computational fluid dynamics (C.F.D) as part of an aerospace engineering degree.  Understandably I was quite excited when my professional assignment was developing a throttle body model for GM’s hardware in the loop system (HIL). Making several assumptions and ignoring the boundary layer conditions I efficiently implemented a 2-d version of the Navier-Stokes equation:

stoked

The model accurately predicted the intake manifold pressure within 0.01% of the measured pressure at a time step of 0.002 seconds.  It used 45% of the HILs processing power (this was in 1995)

Plants in the Field

treeinfield

In the end, I derived a transfer function that modeled the system with 1% accuracy at the same time step using less than 1% of the processor.  That experience, and many since, have caused me to consider the question “what do you need from your plant model?” I have developed 4 basic questions I ask myself and my customers.

What is the required accuracy?

whycantyoudraw_3-2_look_at_things

What is the required accuracy?  In my example above the plant model was connected to a ECU through a 7 bit A/D.  The pressure range was from 1.2 ATM to 0.5 ATM, meaning a resolution of 532 mmHg / 128 = 4.15 mmHg was the best the sensor could do. Therefore from the controllers perspective, anything better than that was not required.  However, the plant model required an accuracy of 0.1 mmHg to maintain stability; a factor of 41 times greater accuracy than required.

There is a second component to accuracy, which is accuracy at the edge points.  A simple polynomial equation could be used to accurately model the throttle body during steady state operation; however during transitions that polynomials accuracy was no better than 10%.

Another example is an engine torque model.  A simple table lookup will provide the engine output torque over the nominal operating range of 800 ~ 5000 rpm.  However, engines do operate outside of that range; the fidelity with which you model those conditions is dependent on what you are validating within your system.

What are my resources?

There are two resources for computations, memory and calculation time (FLOPS for example).  In the example above the accuracy was much greater than required so we considered alternative methods.

The proposed alternative was a 2-D lookup table using two points of past data.  The calculations for the table look up took roughly half the FLOPS of the transfer function and the accuracy was in line with the requirements (1 mmHg).  However, the amount of memory required for the table was approximately 10 times larger than the transfer function(1).

In the end the much higher memory requirements prompted the selection to the transfer function as the solution.

Do you require multiple versions of the plant?

So far in this post, I have talked about targeting a HIL system.  However, it is often the case that a plant may need to be modeled for multiple domains.(2).  In that case developing interfaces to the plant that can be used across multiple environments is an important.  It is now common practice to embed model variants within a closed system to select which version of the plant is required depending

How much do you know about the plant?

The last question to ask in developing a plant model is what is known about the actual plant to enable validation of the plant model.  In some cases, where the physical plant does not yet exist first principal modeling must be done.Even in that, it is often possible to design part of the model based on previous similar plants.  This can be done through the parameterization of the plant model.

Footnotes

(1) In this example, the table data would have only been used by a single function.  If the same table data could have been used across multiple functions then a different decision may have been made.
(2) In a happy ending to my Navier-Stokes model was eventually used as part of another project