In this post I argue for performing simulation instead of model differencing to examine changes to a model. So take a deep breath(0) as I’m about to push this analogy fairly hard… When I’m in the kitchen creating a meal I take care with my ingredients, checking for freshness and the flavor balance (1), always aware of allergy issues(2). I start in a clean kitchen(3), I use a sharp knife and a cast iron pan on a gas stove, quality tools(4). When the dish is done I plate it and I judge it by how it looks and how it tastes. When it is done it is judged by what it is, not what went into it.

By contrast when I balance my checkbook(5,6) in each line item is inspected for correctness and the desirability of the transaction; to understand my finances I need to know each transaction.

A good model is rather like bouillabaisse(9), a complex harmony where the sum is greater than the parts; so then how do you “taste” a model? If you are working in a Model-Based Design environment(10) it means simulating the model and inspecting the outputs of the simulation. If the simulation shows the model is behaving in the expected way for a given scenario, then you know you have the “right” taste. This is a functional review.

“But wait”, you may be saying(11), “what about allergies, a clean kitchen and good tools”? These are addressed by processes and guidelines. Processes protect you from allergies by enforcing guidelines (12), test driven development (13) and development based on requirements (15). The clean kitchen, well that comes from following good processes for an extended period of time; and like any good chief would tell you, “if you are not cooking you should be cleaning”. Good tools, well, see The MathWorks.

“Ok, so I see why be a cook, but why not an accountant? Don’t I need to double check things?” You may be pondering (16). Accounting comes into play if you don’t pass the taste test and if those tests don’t identify the root of the problem. Then performing a block-by-block (or line-by-line) review of model is realvant. Until then doing a model or code differencing does not provide utility.

If I have an equation, say 2+2, and then you change it to 2+3, it is easy for me to see the difference in the outcome by comparing the text. However if this is my equation

The effect of a small change is not obvious by inspection. Differencing text and differencing models is a methodology held over from a time before easy simulation when it was one of the few ways to figure out what was happening. This accounting approach is still valuable as a debugging tool but it is not and should not be your primary method for reviewing models.

Disclaimer: this maybe the first post where the footnotes are longer than the article. I got carried away. I blame Terry Pratchett (17).

Footnotes

This is where I show just how much I can push this analogy:

1. Deep breath: Note if you were in the kitchen while I was cooking that deep breath would be delightful, unless I’m pickling, pickles taste great but brine air is not fine air.
2. Freshness & flavor balance: In software this I validate “am I using the most up-to-date methodologies” and are those methodologies the correct for this project.
3. Allergies: Are a stand in for the know issues in development, either bugs or requirements that need to be met.
4. A clean kitchen: clearly this is a plea to start your development process with a minimum of legacy issues.
5. Quality tools: There is an old cooking saying “The sharper the knife the safer the knife”. Quality tools prevent some category of errors and make it easier to prevent others.
6. Checkbooks: For those under the age of 25, checkbooks are a primitive form of tracking payments in which the writer of a check recorded the amount of the promissory note (check) and subtract that from a banking balance.(7)
7. Checkbooks 2: Technically speaking out online spreadsheet that collates information for statistical analysis.
8. Checks/Balance: This is not to be confused with the concepts of checks and balances that is baked into the US constitution.
9. Bouillabaisse: With the hint of Safron that pulls it together.
10. MBD Environment: If you are not already working in this environment, hopefully this blog gives you reasons to do so.
11. May be saying: I am assuming you are very invested in this blog post and can’t help but verbally exclaim your concerns as you read it. Please be aware of the people around you.
12. Guidelines: Modeling guidelines are like cooking best practices; codified common knowledge.
13. Test driven development: In the same way you press on a burger to know when it is done (14) test driven development makes sure your code is well-done.
14. Press down on the burger: This would be considered black box testing since you are not cutting into the burger.
15. Requirements: Requirements are like recipes; they are what you start with and you may have to evolve over time. Rational evolution leads to tasty dishes, random substitution leads to the garbage disposal.
16. Pondering: Doing this after 11, and noticing the people around you.
17. Terry Pratchett (Sir): https://en.wikipedia.org/wiki/Once_More*_with_Footnotes