Difference reports are a staple of software development, they allow you to quickly see what has changed. Model-Based Design, with its graphical environment, presents some unique issues in differencing; we will examine them with this post.
Differencing best practices
The following best practice hold regardless of the development environment
- Reasonable diffing cadence: Finding the difference between two objects is easy if the number of changes are small; however if large structural, or multiple-point changes have occurred difference operations become difficult if not impossible. Perform differencing operations when no more then 5% of the object has changed.
- Diff function changes / ignore visual changes: Visual changes, such as the use of indents in textual languages, or block placement in graphical, do not impact the behavior of the code. While these changes may violate modeling guidelines they should be ignored during the diff operations
- Include the authors: Have both the original author and updating person take part in the review for complex issues.
- Test after diffing: After performing the difference operations have been run the files should be run through their test suites to ensure that the changes did not have a negative impact.
Don’t judge a model by its cover
In text based languages changes in the text are it; with Model-Based development environments it is the combination of the
- Block connections: How the blocks are connected to each other determines execution order and data flow.
- Block parameters: How the block is parameterized determines the functionality of the block
- Model configuration: The model can have parameters that influence the behavior system beyond the data set by the blocks
Final comment
It is possible to diff the underlying text files that define graphical models; at one point in time this was the only method that was possible. However, doing this presents multiple problems and should be avoided.
- Determining what is significant: The text representations often encode information into parameters / structures. Often it is not obvious what are significant changes versus cosmetic.
- Block changes: The text representation can have the same functionality “shifted” in the code to a new location depending on where it shows up in the model; however this is often not a functional change.
- Interpretation: When reviewing the text changes the reviewers then have to interpret what the text means.
Michael,
Thanks for the post. I’ve seen a lot of people struggle using Simulink diff/merge capabilities…to the point that they go with pure MATLAB or C solutions, avoiding Simulink altogether.
1) The differencing interface is difficult to use, hides quite a few capabilities (filtering, e.g.) and, though it will find everything that changes, sorting through the resulting report is difficult and time consuming.
2) Diffing/Merging Simulink models in a multi-user, collaborative environment is an essential capability. The Simulink diff/merge capability does not seem to scale with even modest-sized teams & models.
3) A common strategy I’ve seen to avoid this diff limitation(?) is to use MATLAB function blocks in Simulink. These MATLAB function blocks contain exactly 1 line of code…a call to a .m file that references other .m files. These text files fit within an existing framework of CM and differencing/merging.
Can you comment on some best-practices for handling these limitations(?)?
Hi Cole,
Thank you for the feedback; I would agree with you that the user interface of the MathWorks model difference tool could be made more user friendly; however once you learn it the environment is quite powerful.
Regarding the other issues you bring up; I would say it falls into two standard problems I have seen
1.) Waiting to long to perform difference tests: in any environment if you wait a long time to perform a comparison the number of changes will grow. Wait long enough and the model comparison tool will have difficulty in determining what has changed and how to best expose the changes. In general I find 2 ~ 4 hours of work on a model is the right amount of time between differencing. (Note: when I say hours of work I don’t mean simulation, rather I mean time spent actually modifying the model)
2.) Running difference tests on to large of a scope: in the same way that you wouldn’t run a difference operation on a 200K line C file you shouldn’t run differencing on a 200K block model. Model differencing should be run on model components (e.g. referenced models and libraries) with sizes in the 1,000 ~ 10,000 block range
As for point 3: having external MATLAB code: that does enable traditional differencing tools but it makes the maintainability of the model much more difficult and it makes some of the built in tool less efficient. Following points 1 & 2 and you should be able to embedded the MATLAB code (when needed) directly in the Simulink model
Another issue people often do not consider are the limitations of textual differencing. Yes, it is easy to see exactly which line(s) of code have been changed. How are those changes coupled to other parts of the code, though? Imagine changing the value of a variable or the method used to compute its value. Are there other parts of the program that relied on the original version of that calculation? One of the really nice features of Simulink is the ease of analyzing and testing these data dependencies through simulation.
By the way, Cole, the XML model comparison tool looks much improved in the R2018b prerelease. MATLAB code in MATLAB Function blocks gets inlined now and the organization of the changes is a lot easier to understand.