A few thoughts on test orginization…

How do you organize your bookshelf at home?  By the author (works well for fiction), by topic (generally good for non-fiction), by size (let’s face it shelf space can be an issue in home libraries.)  Any of these approaches work fine for small libraries, but when the total number of books starts getting large additional information is required to help organize your library.

Image result for library

Metadata and organization

If you have ever used Twitter (disclaimer: I never have) you have experienced metadata in the form of the #ILikeCats or #MyPoliticianIsGreatYoursIsTheSpawnOfSatan.  Metadata is a tag on an object that allows you to augment the information about the object.

Image result for propertiesImage result for metadata

Properties versus metadata

Metadata should not be confused with properties.  Properties are something inherent to the object.  If we extend our book metaphor then we would see

  1. Properties
    • Book title: Nine princes in amber
    • Author: Roger Zelazny
    •  Instance properties
      • Type: softcover
      • Condition: average
  2. Metadata
    • #Genre_Fantasy
    • #Multi_world
    • #Reread

Image result for nine princes in amber

Properties and metadata for tests

Since properties and metadata allow you to organize tests how do I categorize them?  Equally important, what do I not include?

  1. Properties
    • Model name: Name of the unit under test
    • Test harness: Link to the test harness for the unit under test
    • Test name: Short descriptive name
    • Description: Longer descriptive text that summarizes the test
    • Requirement linkages: Links to any requirements covered by the test
    • Data: Input, and expected outputs
  2. Metadata
    • Supported projects:  To support model reuse the tests should tag for which project(s) the test is valid.
    • Supported targets: A list of the targets on which the test is supported.  Such as  SIL, HIL or PIL testing.
    • Test level: An indication of the frequency with which the test is run, e.g. check-in, nightly, weekly, build.  More complex tests that take longer to run should have a higher level.
  3. What not to include
    • Model and data dependencies: These dependencies can be programmatically determined.  Specification of these dependencies will, over time, become out of date.
    • Version history: The version history should be included in the version control software.

Why we care?

In the course of normal development, there is both a local testing step and a global testing step.  At the local level, developers run tests against their updated models.  However, the developers are not expected to know the full scope of the model they are working with; hence the use of metadata and test properties to allow the test environment to fully exercise the model once it is checked in.


By leveraging the test properties and metadata we can more easily reuse tests.  Absent the metadata tests would need to be duplicated across multiple test suites; duplication which can lead to the introduction of errors in the test environment.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.