Levels of testing…

In English, there is a saying, where there is smoke there is fire.  It is an interesting saying which you could take to mean multiple things

  1. Smoke will let you find problems from a distance
  2. Smoke will show up before the fire

At The MathWorks, we call our most primitive tests “smoke tests”.  A smoke test is a basic “does it turn on test”.  The idea behind smoke tests is to validate that the unit under test performs in the most basic fashion, therefore, is ready to run longer more complex tests.  For example, a set of smoke tests for a Model would look like this

  1. Perform an update diagram (Order of a minute)
    if pass then…
  2. Check the model configuration parameters  (Order of a minute)
    if pass then…
  3. Check model for unallowed blocks (Simulink Check)  (Order of a minute)
    if pass then…
  4. Run your complex test suite… (Order of a ???)

Why?

For large, more complex tests, the setup, execution and evaluation time of the tests can be on the order of hours for a complex set of tests.  The ability to “abort” running these longer tests is important as your test suite grows.  You don’t want to take up time on your CI system running tests on models that are not in the correct state.

forestFre

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s