First Principles of Physical Model Verification

Chicken or egg, physical model or prototype hardware? Physical models are used in Model-Based Design to create closed loop simulations to validate control algorithms. To close the loop with confidence the physical model needs to be validated against the real world. And there is the rub; how do you validate a physical model when you may or may not have access to the hardware?

The real world, or something like it…

It is rare that a product is truly “new” and so we can often start off with an atlas(1) that provides initial directions. From there, validating the physical model takes the following series of steps

  1. Determine which variables influence your outputs
    1. Determine which of those variables can directly and safely(2) be measured
    2. Determine the accuracy of those measurements
  2. Determine the operating conditions of your model(3)
  3. Find the “boundary” effects, such as hard stops or friction / stiction interactions.

C3:(4) Collect, clean, correlate!

The first two C’s, collect and clean, are performed on the proto-type hardware or taken from existing research. Once collected, outlying data is discarded and the process of correlating the physical model to the real world data starts.

In an ideal world the inputs to the physical hardware can be directly fed to the physical model and the outputs compared. The model is then “tuned” to reach the highest level of correlation.

Building up from simpler models…

Sometimes the full physical device is not available for data collection. In this case your physical model architecture is built from correlated “submodules.” In this case the accuracy of calculations is even more critical. For the simple case where the accuracy is consistent across the full operating domain it can be calculated as

acc(tot) = acc(1) * acc(2) * … * acc(n)

However, since it is more common that the accuracy is dependent on where in the operating envelope you exercise the model it should be calculated as

acc(tot,r1) = acc(1,r1) * acc(2,r1) * … * acc(n,r1)
acc(tot,r2) = acc(1,r2) * acc(2,r2) * … * acc(n,r2)
acc(tot,rm) = acc(1,rm) * acc(2,rm) * … * acc(n,rm)

In operating regions where sub-models have lower accuracy it is important to maintain high levels of accuracy in the other sub-models.

Footnote

  1. This is both a reference to the image and a chance to say, image searches on the phrase “the real world” bring up the MTV show and almost nothing else.
  2. Remember, one of the reasons for physical models are to take the place of tests in unsafe operating conditions!
  3. Sometimes multiple models are required to handle different operating conditions.
  4. The “3” here was in the mathematical sense, C to the third power. The 4 in parenthesis is the footnote.

Simulating system integration models…

Simulation of system level models is the holy grail(1) of Model-Based Systems Engineering, giving the ability to validate system level functional requirements in a virtual environment. So what is the current state of the system-level simulation? I would say it is looking for its Galahad.

The past, the present

What should you simulate at the system level and what level of detail (fidelity) do you need to achieve? Fidelity limitations limit the answer to the first part; presently the majority of system level simulations run at low levels of fidelity to enable reasonable simulation times(2) and because it’s difficult to validate high fidelity full system level physical models.(3) As a result, scenario based testing(4) compromises the majority of system level testing.

A call to the future

When I first started working, some engineers thought that Moore’s Law would solve the simulation fidelity issue; however as fast as processing power has increased, model complexity has grown as fast or faster; the red queen’s problem. Fortunately there are other solutions.

Divide, smooth, and conquer

Not all of your physical model needs to be high fidelity in the system level simulation; components of different levels of fidelity can be joined together and then “smoothing” operations can be performed on the exchanged data if there are rate or discontinuous behaviors. The full suite of system level models can be comprised of multiple sets of models of different fidelity, allowing for specific types of system level tests.

Avoiding Risk(5)

However the greatest risk to system level simulation is not technical, it is political. Since system level simulation is new to most organizations there may not be an internal champion. Unlike many process adoptions system level simulation by it’s nature requires cooperation across multiple groups. How do you inspire an organization to believe in the holy grail?

  • Start small: While the ultimate objective may be the full system level simulation, errors and improvements can still be found at the subsystem integration level.
  • Identify past integration failures: It is a two step process; first identify the past system integration failures, second demonstrate(6) how system level simulation could have detected the issue prior to deployment.
  • Case studies: While the domain is maturing there is a long history of existing case studies across multiple domains.

Closing thoughts

Ideally system level simulation is started at the start of the project, moving the issue detection upstream. When it is started late in the process it can serve a secondary purpose; providing the tool for root cause analysis for system level bugs. Regardless of where you are in your process it is time to get started.

Footnotes

  1. Like the grail quest, it is never complete but always transformative in a positive fashion.
  2. In general, simulation times of 1/10 of real time are reasonable. For some scenarios slower than real time is accepted to validate critical systems.
  3. Difficulties in validating high fidelity system level models
    1. Practical: Lack of full system interaction data (e.g., how does the AC pump respond to high torque engine loads and high electrical loads?)
    2. Pragmatic: Length of time required to simulate the full model.
    3. Political: Lack of a key stake holder willing to fund the work required.
  4. Scenario based testing is an excellent method for detecting system level faults for intermittent messaging, e.g. what are the effects of system latency in the communication of critical data.
  5. The last image was not an endorsement of the board game Risk.
  6. System level failures are easy to show when you have full test logs of existing products “in the field.” A static playback of the data can be provided to stand in for the model

DevOps

Before we dive in, I will paraphrase my wife’s late grandmother who lived to 102. When asked why she seemed to have so few “disagreements” with people she answered “I make sure I know what they are asking before I answer.”

The last ten years of helping companies adopt processes has shown me that 90% of all successful processes are anchored in “we understand what the other person is asking so we understand what they need.”

A snapshot of DevOps

From the Software Design V to….

Over the last 25 years the Software Design V has been the default graphical metaphor used to explain the software release process. Down from requirements to implementation, the V ramps up through testing, and finishing at release. Early V was strongly linked to the waterfall development process. As criticisms of waterfall development processes emerged, the V adapted by adding “feedback loops” between parts of the V. As agile emerged the V continued to evolve; agile’s small fast development cycles required smaller faster V and we entered into the ΔV territory. In the last 5 years with increasing prevalence of over air updates and development in the cloud, developers’ needs “broke” the V.

New shape, same objective

If you read a list of the benefits from adopting DevOps you could be excused for thinking they were the benefits of an Agile, Design V, or even a Waterfall development process.

  • Improved reliability
  • Increased scalability
  • Simplified & improved collaboration…

The fact there is so much overlap is not as a criticism of the DevOps, it is the recognition of the underlying purpose of all processes: promote clear communication between the stakeholders in the process.

So why do we need DevOps?

There are more lines of code in a modern dishwasher than in the entirety of my first car.

But it isn’t the size of the code that drives the need for DevOps.

The mathematics equations associated with a LIDAR system outstrips the complexity of the Apollo moon shot by multiple orders of magnitude.

But it isn’t mathematical complexity that slings us round towards DevOps.

The simulated hours of testing for a robotic vacuum cleaner is greater than the total time in pig models for the first artificial heart.

But it’s not the number of hours spent in simulation that is primed the pump in favor of DevOps.

So why do we need DevOps?

It is about clarity of communication. In all of the cases I listed above it was “like talking to like”; controls engineers working with controls engineers, AI people with AI people, testers with testers. The need for DevOp is driven by the interaction of groups that have not historically worked together. DevOps becomes the common language and process bridging the groups.

Feedback loops and continuous integration

The brilliance of DevOps is that it found a point of intersection between the deployment and controls groups. The process connects the concepts of Feedback Loops (controls) and Continuous Integration (development): the “X” point of the DevOps cycle. It is at that point that a common ground and common understanding is easiest to achieve.

X-marks the spot where the treasure of DevOps is found

This brings us back to the start of this blog; when you are trying to solve a problem you need to understand what the other person is saying. As the software release and update processes become more interconnected across multiple groups, the way we solve them is through the implementation of a common process and vocabulary

Large engineering projects are always easier if you speak the same language

Short Blog Bits

In this post I want to do something different; a collection of a few “basic” tips for system software, brought to you in folk wisdom format (alternating, bird, craft, bird, craft, ending on horse, of course).

Six of one, Half a dozen of another

How data is specified makes a difference. In this case “6 of one” is a direct specification of the number. “Half a dozen of another” is a calculated value which requires machine operations. When possible use direct specification of data.

A stitch in time saves 9

This is a tricky situation; preventive maintenance will almost always have downstream benefits. At the same time if you are spending time looking for the “single stitches” to be corrected you are taking time away from the core work that needs to be done. Context switching between tasks needs to be taken into account when performing corrective tasks; finish what you start, don’t ignore small problems.

A bird in the hand is worth two in the bush

Project planning is about project completion; “feature creep” is a common problem when the focus isn’t on getting (and maintaining) the “bird in hand.” A bird in the hand is wonderful, a bird at home letting you get ready to go get more birds is best.

Measure twice, cut once

With Model-Based Design or any software development process, the mantra should be measure as you go, work to refine. Software development can be seen as a series of tasks from roughing it out with a saw to fine tuning with sandpaper. You should always be measuring to know where you are.

Never look a gift horse in the mouth

Legacy software can be seen as a gift horse; it may let you ride into town but if it is knackered you will need to become the knacker, adjust and replace it. Always review your legacy software; you need to “look it in the mouth.”

Cooking Thermodynamics

This year for Christmas, knowing my love of comparative cooking methods, my wife Deborah got me an Air Fryer; the results have been delicious.(1) This got me thinking about the thermodynamics of cooking(2) and what it would take to model both an air fryer and a past present (the instant pot).(3)

Physical (modeling)

Cooking is a heat transfer problem; move the energy from the cooking device into the food.

  • Pan cooking: conduction
  • Oven cooking: conduction and radiation
  • Air frier: conduction, radiation and convection
  • Instant pot: conduction

However there are limitations. The maximum temperature of an oven is limited to the energy of the heating method (gas or electric), boiling liquids are “capped” at the boiling point of water; this then is the “trick” to both the air fryer and the instant pot. The air fryer combines 3 methods for transferring heat, while the instant pot increases the pressure in the “pot” which raises the boiling point of water.(4)

Three simple rules

3 rules

Cooking with an air fryer has three simple rules:

  1. Pre-heat: Heats the basket, giving conduction cooking
  2. Space out: Aids in convection
  3. Shake: Gives all sides a shot of conduction

From this we can determine that when you cook there is browning in the contact in the pan, that the air blows around the food from all sides (and the bottom) and that to get brown all sides of your food need to touch the bottom. So what would the physical model look like and how do we optimize the results?(5)

ParameterEffectTrade off
Pan thermal massBrowning effectLonger warm up time
Grate arrangement of panAirflow (convection) between foodReduces thermal mass
Blower speedIncrease convectionEnergy usage
Food packingImpact convectionVolume of food to be cooked
at one time
TemperatureRate of food temperature increaseEnergy usage

Equation time!(6,7)

For now let’s assume a Platonic solid French Fry, e.g. cube 1 cm x 1cm x 1cm.(8) In this case the equations for the heat transfer could be written as…

  • Conduction: Q/Δt = -k * A * ΔT/Δx (9)
  • Convection: Q/Δt = A h(Tp – Tpf)(10,11)
  • Radiation: By evaluation O(10-3): dropped

By further inspection we can see that Q is the energy from the heating unit. If we assume that the thermal mass can be treated as an infinite sink (and if we have Platonic Fries we can allow this) then we can reorder the equations as so…

Tpf = Q/(Δt) * ((1/k*A1) + (1/h*A2)) + Tp

The Platonic Fry Model

Extensive field work is now ongoing to determine by experiment the values of k,h, A1 and A2 for Platonic fries (and other foods). The formula for “browning” is also under development, but as I hope this article has shown you, it is possible to determine through analysis and simulation.

Footnotes

  1. The biggest surprise is how good the “air fried” carrots are; they are a reliable “go-to” afternoon snack.
  2. The control algorithm for this device is very simple, a boolean for the fan on or off, and a predictive element for the temperature.
  3. I find it most interesting to alternate between new kitchen gadgets and experimenting with just a pan, bowl and knife.
  4. I just had a thought; when you flambe food or cook with alcohol in general it boils off due to the lower boiling point than water; what would happen in the instant pot?
  5. In this case optimizing the results means crispy exteriors and internal temperature cooked to “done.”
  6. “Equation time” sounds like a kid show that I would like to see produced.
  7. Despite the image we will not be covering solar roasters today.
  8. When I picture this it sounds more like Platonic Hash Browns
  9. The thermal conductivity of a potato should be similar to that of water due to the high water content of the potato.
  10. Tp is the pan’s temperature and Tpf is Platonic fry’s temperature.
  11. The “A” term in this equation is affected by the crowding of the pan; the more crowded the food the lower the A.

Testing Requirements: Part 3

In part 3 of this series, we look at how and why we create test harnesses.

Post credit thoughts

MathWorks provides stellar documentation on the creation of test harnesses; the key thoughts should be on why we create them (to avoid modifying the UUT) and how we call them (from test runners!)

Types: Not just for fonts…

Typographically inclined people can debate for hours the importance of font selection in the presentation of a message; a proper font(1) sets the right type of message.(2) With programs, data type is used to specify range and resolution of the data as well as determining the total memory and speed of your program.

I is for integer, at least in FORTRAN

In modern programing languages there are three primary data formats; integer, fixed point(3) and floating point. The range of integer data types is determined by the size of the data, e.g. 8, 16, 32 or 64 bit data types and the signed / unsigned nature.(4)

Floating point data represents information as a combination of a sign bit, an exponent component and a fractional component. This gives floating point data a greater range and resolution; however this can lead to issues when performing operations on data of significantly different orders of magnitude. For an overview of floating point data I would recommend this Wikipedia entry.(5)

What is the “point”?

Memory usage is easy to understand,(6) using all 32 bit integers in your program will use twice the memory of 16 bit integers; the same idea applies to floating point data types. The trade-off then comes with accuracy and speed. Integer based arithmetic is computationally faster then floating point; likewise, 64 bit floating point takes longer then 32 bit. The objective should be to use the smallest data type that fully represents your data after completion of the operation.

Tips and tricks!

  • Overflow, underflow and precision loss can be detected through the use of simulation or tools such as Polyspace
  • Disable range checking and overflow options for generated code to create more efficient implementations

Footnotes

  1. Here in this blog I lack the control of what font you see in your browser or in your email; for all I know you are using Wingdings (🕈︎♓︎■︎♑︎♎︎♓︎■︎♑︎⬧︎).
  2. And of course typesetting is part of the act of printing, the precursor to publishing electronically.
  3. Fixed-point could be considered a special case of integer data type, we will look at it in a future blog post.
  4. E.g. an 8-bit unsigned integer would have a range of 0 to 2^8 (256), while a signed 16-bit integer would have a range of (2^15 – 1) to 2^15 (-32767 to 32768). In every case they have a resolution of 1.
  5. There is a conceptual overlap between fixed point data floating point operations work; this link provides a general overview of fixed-point data.
  6. There is a caveat here; all hardware has a “smallest data type.” If you specify a data type smaller then the smallest data type you do not see any memory savings.