Does A == C?

I recently taught a class on testing fundamentals, in it I made the comment that by my estimate there are over 8,000 lines of code in MATLAB dedicated to the simple (simple seeming) test

A == B

Why?  What is testing for equality so hard?  Let’s break it down.

  1. Data types and numerical precision: depending on the selected data type the resolution to determine “equal” may not be present.  You can end up with false positives and false negatives
  2. Tolerances: You can take data type into account by adding tolerances into comparison.
    1. Absolute tolerance: abs(A-B) < tol
    2. Relative tolerance: abs(A-B) < per_tol * A
  3. But what about zero: Percentage tolerance is good, but what do you do when the value is zero?
    1. Relative tolerance (mean): abs(A-B) < per_tol * mean(A);
    2. Realitive tolerance (max): abs(A-B) < per_tol * max(A);
    3. Realitive tolerance (moving average) : abs(A-B) < per_tol * mean(A(i-N) : A(i+N))
  4. What about noise: for measured data how do you handle the “junk data”?
  5. What about missing data: much like junk data what do you do with missing data points?
  6. What about data shifts (temporal or other): it is fairly common for comparison operations to take place with “shifted” data.  Where one signal is offset by some fixed amount in time.
  7. What about non-standard data formats: how do you handle the comparison of a structure of data?  Do all elements in the structure have to match to “pass”?   Do you apply the same standard of tolerances to all elements?

Image result for pass no pass

You can quickly see where my estimate of 8K lines of code come from.  Why then do I mention this?  Two reasons

  1. Start thinking about the complexity in “simple” tests
  2. Stop creating test operations when they already exist

Image result for reinventing the wheel

Endnote

This is written in the context of testing.  Any sort of algorithmic or logical code will, of course, use comparison operations.  For those cases keep 2 simple rules in mind

  1. Do not use floating point equivalence operations:
  2. Take into account the “else” conditions

 

 

One thought on “Does A == C?

  1. Does A= =C?
    You mean check that A = C? OR check that check that A = = C works?

    From 40 years ago, I seem to recall at least these ways to specify test cases.
    Internal testing
    Test every possible path through the code from start to end (impossible given any but a trivial algorithm).
    Test every line is executed correctly at least once.
    External testing
    Test cases feature lowest, middle and highest values for each variable (and invalid values?).
    Test cases are built by users and/or domain experts.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s