There is an old saying that “You can’t compare apples to oranges”; but frequently as an engineer, one of our jobs is to compare apples to oranges. Why and what does that mean?
Frequently, when systems are built the actual system does not yet exist. As a result, we need to draw analogies between the two items. So if we are going to compare apples and oranges how do we “juice” them for the correct data?
The mapping between objects: first principals
One objective with these mappings is to reduce the volume of first principals modeling that needs to be done; e.g. instead of building up a complex system of equations, use a modified data reduction of an existing system modified for the new system. However, to do that, we must first validate that the fundamental behavior of the systems scale between each other. For example, scaling a 4 cylinder 1.8L engine to a 2.1L engine is straight forward. To go from a 4 cylinder to an 8 cylinder is also straightforward if a slightly different problem.
What is the same? What is different?
You selected the reference model because there are things that they share in common. At the same time, there will be things that are different. In our example of the 4 cylinder to the 8 cylinder engine, there are differences in total displacement (an easy mapping) torque output excetra. The thing that may not be easy to map is the change to the in the vibrational aspects of the engine. The questions to ask are
What physical aspects of the system am I concerned with?
Torque
Fuel consumption
Max rpm
…
What aspects am I not concerned with?
Vibration
Heat transfer
….
If you find this content useful, please consider subscribing
So my first step into machine learning is examining supervised learning and my old friend curve fitting. This was a fun trip down memory lane as my numerical mathematics background had me well prepared for this topic. Further, in the 25 years since I graduated, the basic tools for performing regression analysis have greatly improved, I was able to use the built-in curve fitting tool in MATLAB to perform these operations directly. At the same time, I quickly remembered the “gotchas” about regression…
Starting points
As a starting point, I thought I would see how well the tool would do with a known polynomial equation. I defined a simple 3rd order equation and added some noise into the signal. I then ran the results through the curve fitting toolbox…
The data
As a sanity check, I first tried it out using the “real” X and Y data. As would be expected the tool spits back out the coefficients that I expected. When I ran it with the noise (and telling it to use a 3rd order polynomial) it returned coefficients fairly close to the “real” things.
Curve fitting within the boundaries
Coefficient
Original
Regressed “Real”
Regressed “Noise”
P4
0.0
-3.79e-14
-0.745
P3
1.5
1.5
3.05
P2
1.2
1.2
0.8709
P1
-0.5
-0.5
-0.522
R-Square
NA
1.0
0.9939
Within the defined range we have an R-squared value of 0.9925, pretty darn good. But, the question quickly becomes, how does it hold up outside of the known range?
Pretty well actually?
So depending on how you want to look at it, as an absolute error or a percentage error you may be doing “O.k.” or not. But what if I didn’t know what the underlying equation was? A power function would seem like a reasonable. If I try a first order power equation
y = a * exp(b * x) or second y = a * exp(b * x) + c * exp(d * x);
Order
R-square
1st
0.9852
2nd
.9961
Within the original bounds, the R-Squared value is close to the actual polynomial. If we used the 1st order equation just in that region it executes faster then the 3rd order polynomial so we may want to go with a “fake fast fit”. (say that three times fast).
Why do we try to “fit-in”?
There are several reasons why regression is performed. Sometimes it is because the underlying equations are unknown, are too computationally expensive, there isn’t an equation. Regardless of the reason when I am trying to perform a regression I perform the following steps
What are the fundamental physics: Use this to pick out which terms and functions to use. (Note: The curve fitting toolbox has an wonderful “custom equation” option that let’s you try these equation based approaches out)
Boundaries? I try to understand how the data outside of the known data set? In my example the exponential function does not perform well much outside of my known data.
Roughly 30 minutes after I finished writing this blog post I stumbled across this post “Explore Runge’s Polynomial Interpolation Phenomenon” by Cleve Moler. I would be amiss to not include this link for three reasons
Cleve’s post clearly articulates the mathematical underpinnings of interpolation
As the creator of MATLAB, his work is why things are “greatly improved”.
It’s a post about Runge, and since I had a cat named Runga-Kutta (RK) back in college well…
Warning: Bad aerospace engineering pun follows:
When I was in grad school and had that cute cat I was working on a simple project, a device to determine the terminal angle (stall) for a kite. For the most part, I worked on this project in my apartment, and Runga loved to attack the kite’s tail, jumping at it attacking it. That was well and good in the apartment.
Well at the early testing stages I was flying the kite off of my back deck. One day while flying RK managed to get out. In a leap of excitement she jumped onto the kite’s tail (knowing from experience she would land happily on the carpet). Except, this time, the kite was 8 feet off the ground. Needless to say, both the cat and the kite crashed to the ground proving that for this case the Kutta condition did not hold.
If you find this content useful, please consider subscribing
For the last two years, I have written about Model-Based Design in the context of classical control algorithms and software verification processes. Over the next year, I am broadening the scope of this blog as I investigate how machine learning can be applied to MBD in the context of safety critical systems.
My questions
What is the current state of the art in Machine learning?
How can it be used for controls problems?
When should it be used? When should it not be used?
Where is it the only solution?
What are building blocks of machine learning?
Starting small, what are the easy things to do?
Hitting the middle, what is a “real” project?
Common “starting” pitfalls
Testing and verification
Can you do black box testing?
How do you test an “open world” model?
So join me on this trip…
I will share my insights, my mistakes and look forward to hearing from you…
In English, there is a saying, where there is smoke there is fire. It is an interesting saying which you could take to mean multiple things
Smoke will let you find problems from a distance
Smoke will show up before the fire
…
At The MathWorks, we call our most primitive tests “smoke tests”. A smoke test is a basic “does it turn on test”. The idea behind smoke tests is to validate that the unit under test performs in the most basic fashion, therefore, is ready to run longer more complex tests. For example, a set of smoke tests for a Model would look like this
Perform an update diagram (Order of a minute)
if pass then…
Check the model configuration parameters (Order of a minute)
if pass then…
Check model for unallowed blocks (Simulink Check) (Order of a minute)
if pass then…
Run your complex test suite… (Order of a ???)
Why?
For large, more complex tests, the setup, execution and evaluation time of the tests can be on the order of hours for a complex set of tests. The ability to “abort” running these longer tests is important as your test suite grows. You don’t want to take up time on your CI system running tests on models that are not in the correct state.
If you find this content useful, please consider subscribing
