Tag: #foundation

Modular testing environments

Published on by Michael Burke2 Comments

Foundations define and limit the structures we create; this is as true in Model-Based Design as it is in architecture.  With that in mind, I want to use this post to discuss the concept of modular testing environments (MTE).  First, I will point to an earlier blog post “Testing is software“, before I drill deeper into the concept of MTE.

What is a modular testing environment?

A modular testing environment consists of 5 parts

  1. Test manager: a test manager provides the framework for running, evaluating and reporting on one or more test cases. Further, the test manager provides a single hook for the automation process.
  2. Test harnesses: a test harness is the software construct that “wraps” the unit under test.  Ideally, the test harness does not change the unit under test in any fashion; e.g. it allows ‘black box’ testing.
  3. Evaluation primitives: the evaluation primitives are a set of routines that are commonly used to evaluate the results of the test.  Evaluation primitives range from a simple comparison against an expected value to complex evaluations of a sequence of events.
  4. Reporting: there are two types of reports, human and machine readable.  The human readable reports are used as part of the qualification and review process.  Machine-readable reports are used for tracking of data across the project development.
  5. Data management: testing requires multiple types of data, inputs, outputs, parameters and expected results.

Why is a modular testing environment important?

Having helped hundreds of customers develop testing environments the 5 most common issues that I have encountered are

  1. Reinventing the wheel, wrong:  Even the simplest evaluation primitive can have unexpected complexities.  When people rewrite the same evaluation multiple times mistakes are bound to occur.
  2. Tell me what happened:  When tests are pulled together in an individual fashion it is common for there to be limited or inconsistent reporting methods.
  3. Fragile tests: A fragile test is one where if the inputs change in a significant fashion the test has to be completely rewritten.
  4. “Bob” has left the company:  Often tests are written by an individual and when that person leaves the information required to maintain those tests leaves with them.
  5. It takes too much time:  When engineers have to build up tests from scratch, versus assembling from components, it does take more time to create a test.  Hence, tests are not written.

Final thoughts

Verification and validation activities are central to any software development project, Model-Based Design or otherwise.  The easier you make the system to use the more your developers will embrace them.

Collecting feedback…

Published on by Michael Burke1 Comment

Please forgive the early post…

When developing a control system feedback is critical; in creating a company wide software proces feedback (from your employees) is even more importaint.  What is the best way to gather that information and what is the information that you should be collecting?

AAEAAQAAAAAAAAIUAAAAJDM0YTBmOTg1LTFlY2MtNGI0MS1iNWJiLTcwNWY5NmRmZWExNQ

What did your bug reports tell you?

Bug tracking systems serves as the “first pass” for information reference.  When developing the software process a category of “workflow issues” should be included in the tracking software.  These workflow bugs will show problems related to

  • Poor documentation: The primary way users learn about the Model-Based Design workflow is through the documentation.
  • Architecture interfaces: Poor interfaces, either for model or data integration will emerge as new design patters are exploreed by new groups.  The process adoption team must determine if the interface should be extended or a new interface defined for the group specific requirements.
  • Test failures:
    • Modeling guidelines: Failures in modeling guidelines will show where users have difficulty in conforming to modeling standards.
    • Regression tests failures: These can indicate an improperly defined regression test system.  During the inital development of the test environent it is common for there to be errors in the system.

bugReport

Direct feedback / viewing

At the one, two and six month marks groups new to the process should be brought in for a formal process review meeting.  During the meeting the following activities should take place.

  • Design reviews:  The models, tests and data managment files should be reviewed to ensure that best practices are followed.
  • Pain points: Request feedback from the teams to capture existing pain points.

Final thoughts

Collecting feedback from new teams is critical understanding where processes can be improved.  The development is as always an iterative process requiring input from teams outside the inital “core” team.

If / elseif/ else: Why didn’t you ask me that in the first place?

Published on by Michael BurkeLeave a comment

Conditional logic, such as if/else, switch/case constructs, truth tables and state machines are common to most programming languages.  In today’s blog, there are two aspects of conditional logic that I want to address in today’s post,  independence, efficiency versus clarity.

Independence

Let’s compare two simple examples, in the first we have a single variable “A”, in the second we have three variables “A”, “B” and “C”

iflogicelseifstartingpoint

If we consider the first case the order of the comparisons (A==1) and (A==2) can be changed with no impact on the functional behavior of the code.  This is because the two conditions are mutually exclusive (1).  However in the second case, without formal analysis,  we cannot say if the order affects the functional behavior.  That is to say, if both A, B, and C can be true at the same time then the order of evaluation impacts the resulting functional behavior (2).

Frequently people are not aware of this possibility and as a have difficulty in evaluating the functional behavior of their code(3).

Efficiency versus clarity

For most people, the standard gear shift in cars is well known, with the order of states PRNDL as a standard (Park,  Reverse, Neutral, Drive, Low).  Therefore you commonly see conditional logic (or state logic) that represents the PRNDL like this…

prndl

However, from an efficiency point of view, this is poorly defined.  The majority of the time the transitions you will see are from park to reverse, park to drive, drive to park or reverse to park.  This would suggest the following organization

prndl_eff.

Why is this more efficient?  Simply put it reduces the average number of comparison operations required to reach the desired end state.

Compound conditions

The previous example was fairly straight forward; there is a single variable with independent states.  It is more common to find compound conditions defining the if/then/else logic.  Let’s look at our friends, A, B and C again.

compoundif

This first example is a classic compound if statement.  In this example, the logic is fairly straight forward.

deeplogicSo which of these is more readable?  This is a difficult question to answer.  In this case, the indented format of the code makes the binary nature of the if/else-if/else conditions clear.  However, if I introduced more complex evaluations, non-binary operations, then it would perhaps a hybrid of the two formats would be preferable.

Rules of thumb

  1. Keep the number of comparisons on a single line 3 or less:  Exceeding 3 comparisons per line make validation of the coverage of the case difficult.
  2. Keep the depth of if/if/if trees to 3 or less:  Traceability back to the top of the tree decreases at depths greater than 3.
  3. Place the binary comparisons at the top level of your if/if/if trees:   Placing the binary operations at the top of the tree reduces the overall number of if/else required.

Final thoughts

Formatting and density of information are frequently “hot topics” of debate.  The rules of thumb listed above should not be considered a final verdict but rather a starting point for discussion.  As always I welcome all comments.

Footnotes

(1) Note this may not be true if you move to a quantum computer where a single bit could maintain multiple states.  Until then you are fine this statement holds true.

(2) The second screen shot has a common error; lack of documentation.  The code should be commented to state if the code can be reordered.

(3)The common scenario has a person perplexed why they do not call “callBsFunction” when B is true.  Not realizing that it is gated by the “A” is true if statement.