Anyone who has worked with software for more than 3 years knows that migration between software releases is a fact of life; having that process be smooth and easy is not always a fact of life (anyone remember Windows ME pains?)

Making migration easy(er)

One of my early swimming coaches was famous for saying “you win the race by how you train.”  I have found this advice to be true in most aspects of my life.  Projects succeed or fail based on the preparation you do as much as your execution.

Preparing for migration

In preparing for migration I start by asking 3 questions

1. What things are we doing now that are working well?
2. What things are we doing now that are hard to do?
3. What things do we want to do that we can’t do now?

The first question focuses on maintaining current functionality.  The second and third look at how to make things better.  Improvements to processes can be made either through refactoring of existing processes (or creating new processes) or through the adoption of new tools.
One of the critical things to keep in mind with software upgrades is that it is not just changing tools.  It is, or should be, about changing processes.  [Note: for minor migrations of a single tool the associated processes may or may not require updates.]

A few thoughts on type two problems

The “type 2” problems, “what things are we doing now that are hard to do?” can be further thought about in a few components.

• The process runs slowly:  Frequently, but not always, upgrades in software can provide an increase in speed.  Additional, process changes, may provide speed improvements.
• The process is complicated to execute:  Complex processes can be difficult to execute. Often complex processes were developed due to limitations in the tool when they were initially developed.
• The process has bugs: Before upgrading software validate that the bugs in the software have been resolved.

The more things change the more they stay the same…

When you upgrade you still want some things to be static: your results.  The best method for ensuring that your results (deliverables, code,…) remain the same is by developing test cases that “lock down” your deliverables.

When comparing test results between different tools there are a couple of things to keep in mind.  First, for every test as “acceptable” change should be defined as there may be small deviations which have no effect on the overall systems performance (though for some tests no change will be allowed.)  Second, in some cases testing in newer versions of software bugs that were not detected before may be uncovered.

Testing the testing environment

As a last note; if as part of your migration you are are updating your testing environment then you need to validate the behavior of your testing environment.  This is generally done through manual inspection of a subset of the full test suite.  The key factor is to have a subset that contains a full set of all types of tests are performed by the testing environment.