When it comes to safety standards, such as the ISO-26262, the old adage “better late than never” can be both dangerous (1) and costly. The simplest, somewhat humorous, description of a safety critical standard that I have read is the following
- Say what you are going to do
- Do what you said you would do
- Verify that what you did matches what you said
- Generate reports
While it is fictitious in its simplicity, it gets to the heart of the matter. Safety critical processes are about being able to show that each step along the development path you both plan out what you intend to do and verify that what you did matches your intentions (plan, do, show).
If that is all
You have to do, how hard can it be? Well first, let’s talk about what it takes to “show” that you did what you said. First you need to be able to show traceability between all artifacts. This means you need a robust tool that will show the link between
- Requirements to the model
- Requirements to tests
- Test results to the model
- Test results to the requirements
- The model to the generated code
- Integration tests to requirements
- And between all the other components
Furthermore those links need to take into account the version control for all the units under test. Setting up the “hooks”(2) for all of these components is a task that needs to be done at the start of the project. There is a reason for the old joke about airplane development: “For every pound of plan you have 20 pounds of paper”(3)
Now that you have your “outline” what now(4)
Tracing the steps is just the “first” step; next, you need to validate the behavior of every tool (including your tracing tools) along the way. There are 5 basic steps for validating a software tool
- Create a validation plan: Define what it is you will be testing, under what conditions, what is the environment, who will perform the validation…
- Define system requirements: The creation of a system requirements (SRS) document breaks down into two parts; infrastructural and functional. During this time a system risk analysis document is created along with mitigation strategies.
- Create a validation protocol and test specs: Definition of both test plan (how you will test) as well as the specific test cases. The creation of a traceability(3) matrix between the plan and the test plan is created.
- Perform the testing: Execution of the tests defined in step 3
- Review and update: Collect the results of step 4, for any issues that failed the validation plan, determine if the fall under the mitigation strategies or if the plan / tool needs to be updated.
Getting your ducks in a row
Assuming you get all your ducks in a row (6) what next? The next step is to roll out how you will use those tools to your end users. Part of the software validation process is specifying how the tool is used; this can take the form of modeling guidelines (MAAB Style guide), defined test frame works or other workflow tools.
The only time is NOW
As these tasks start mounting up you can see why the “better late than never” will not work for a safety critical workflow; by the time “late” comes along, you have already been developing algorithms without the guidelines, creating artifacts without traceability and using tools that may, or may not, be certifiable.
There is, of course, good news. Processes learned in one project can be reused (as can verification artifacts) so much of what you a face is a one time upfront cost. If this is your company’s first time, you can also leverage industry best practices such as the IEC Certification Kit.
Footnotes
- People often talk about things being dangerous that are not, in fact, dangerous. However when it comes to safety standards, the failure to start early can result in critical mistakes entering into the project which can lead to injury and even death.
- Hooks is a term used to describe the infrastructure to connect different components together.
- A joke about “paper airplanes” would make sense about now.
- Since an “outline” is a “tracing”
- The longer you work in the safety critical world, the more often you will hear the term “safety critical”
- And for a software development project there will be multiple “ducks” to validate from Test vector generators, code generators, compilers, test harness builders….