Tag: #machineLearning

Math! Solving problems, making friends…

Published on by Michael BurkeLeave a comment

A few months ago there was an interesting article about AlphaGo and deep learning.  What made the solution interesting is that AlphaGo determined a new strategy for playing Go, e.g. it wasn’t just replicating the best of current players, it was creating something new.

Image result for AI something new

So this got me thinking about sensor placement. Let’s say you are building an autonomous vehicle; you give it radar, LIDAR, cameras, heck even a person walking 5 feet in front of the car with a red flag. But how do you know what the best configuration of those sensor is? This is where math and deep learning can come to the rescue.

It’s research, dam the budget!

With research vehicles it is common to go high end with the sensors and processors. Often the number of sensors are far in excess of what the final vehicle will have; the question then becomes “If you train your controller using 10 sensor and the final vehicle will only have 3 do you have to retrain?”

So here is the insight, in regression there is the concept of a synthetic variable. E.g. you can take the raw data X and put X^2 into the equation. What if, for sensor data, you optimize for sensor position?

Image result for multiple angles
Trig and Geometry!

Step 1: Train your model using the full set of sensors. Train it until you reach the highest confidence level you can.

Step 2: Define the set of equations that will take your original sensor data and use them to create “artificial” sensor at arbitrary locations. E.g. if you have 10 LIDAR sensors, then it could be that
AS_1 = func(X1,Y1,Z1) = func(L1,L2,L4,L6)
AS_2 = func(X2,Y2,Z2) = func(L3,L4,L7)
AS_3 = func(X3,Y3,Z3) = func(L2,L6,L9,L10)

Step 3: Using the already trained data train the new sensor array

Step 4: Optimize the models as a function of the sensor placement and number of sensors

Sensor fusion before the sensors…

I think of this approach as “sensor fusion before the sensors”. What is interesting about this approach is that it is possible that we could discover a combination of sensors (and yes this should be done with multiple sensors) that has a higher accuracy and lower costs than we expect.

Image result for sensor fusion

If you find this content useful, please consider subscribing

The truth about regression…

Published on by Michael BurkeLeave a comment

So my first step into machine learning is examining supervised learning and my old friend curve fitting.  This was a fun trip down memory lane as my numerical mathematics background had me well prepared for this topic.  Further, in the 25 years since I graduated, the basic tools for performing regression analysis have greatly improved, I was able to use the built-in curve fitting tool in MATLAB to perform these operations directly.  At the same time, I quickly remembered the “gotchas” about regression…

Starting points

As a starting point, I thought I would see how well the tool would do with a known polynomial equation.  I defined a simple 3rd order equation and added some noise into the signal.  I then ran the results through the curve fitting toolbox…

The data

As a sanity check, I first tried it out using the “real” X and Y data.  As would be expected the tool spits back out the coefficients that I expected.  When I ran it with the noise (and telling it to use a 3rd order polynomial) it returned coefficients fairly close to the “real” things.

Curve fitting within the boundaries
CoefficientOriginalRegressed “Real”Regressed “Noise”
P40.0-3.79e-14-0.745
P31.51.53.05
P21.21.20.8709
P1-0.5-0.5-0.522
R-SquareNA1.00.9939

Within the defined range we have an R-squared value of 0.9925, pretty darn good.  But, the question quickly becomes, how does it hold up outside of the known range?

Pretty well actually?

So depending on how you want to look at it, as an absolute error or a percentage error you may be doing “O.k.” or not.  But what if I didn’t know what the underlying equation was?  A power function would seem like a reasonable.  If I try a first order power equation 

 y = a * exp(b * x)
or second
y = a * exp(b * x) + c * exp(d * x);

OrderR-square
1st0.9852
2nd.9961

 Within the original bounds, the R-Squared value is close to the actual polynomial.  If we used the 1st order equation just in that region it executes faster then the 3rd order polynomial so we may want to go with a “fake fast fit”. (say that three times fast).

Why do we try to “fit-in”?

There are several reasons why regression is performed.  Sometimes it is because the underlying equations are unknown,  are too computationally expensive, there isn’t an equation.  Regardless of the reason when I am trying to perform a regression I perform the following steps

  1. What are the fundamental physics:  Use this to pick out which terms and functions to use.  (Note: The curve fitting toolbox has an wonderful “custom equation” option that let’s you try these equation based approaches out)
  2. Boundaries? I try to understand how the data outside of the known data set?  In my example the exponential function does not perform well much outside of my known data.
Image result for fitting in
https://www.tor.com/2018/10/24/fitting-in-max-gladstone/

Disclaimer:

Roughly 30 minutes after I finished writing this blog post I stumbled across this post
“Explore Runge’s Polynomial Interpolation Phenomenon” by Cleve Moler.  I would be amiss to not include this link for three reasons

  1. Cleve’s post clearly articulates the mathematical underpinnings of interpolation
  2. As the creator of MATLAB, his work is why things are “greatly improved”.
  3. It’s a post about Runge, and since I had a cat named Runga-Kutta (RK) back in college well…

Warning: Bad aerospace engineering pun follows:

When I was in grad school and had that cute cat I was working on a simple project, a device to determine the terminal angle (stall) for a kite.  For the most part, I worked on this project in my apartment, and Runga loved to attack the kite’s tail, jumping at it attacking it.  That was well and good in the apartment.

Well at the early testing stages I was flying the kite off of my back deck.  One day while flying RK managed to get out.  In a leap of excitement she jumped onto the kite’s tail (knowing from experience she would land happily on the carpet).  Except, this time, the kite was 8 feet off the ground.  Needless to say, both the cat and the kite crashed to the ground proving that for this case the Kutta condition did not hold.

Image result for terminal angle stall

If you find this content useful, please consider subscribing