Recent studies (1) suggest one reason for the waves of death of mega fauna can be attributed to the arrival of humans into the ecosystem. Large animals that could thrive in competition with each other quickly succumb to the fast moving ape with a mind. The COVID-19 event is a trigger for multiple changes in how we develop software; to paraphrase, the apes are coming for the mega models with the tools the monolith provided.
What is a “large model”?
I have written before about how to measure the size of a model. Today I want to focus on what it means to be a “right-sized” (2) model and what it takes to realize it in practice. First off, going smaller does not directly correlate with less work; doing something well in a small space is harder. (3) (4) Second, clarity should, rarely (5) be sacrificed for size. With those thoughts, we will tackle “right-sized.”(6)
I am working from the following axioms driven by COVID
- Fewer “informal” review points(7)
- Greater reliance on Simulation based verification(8)
- Use of coverage metrics in reviews
- Use of specification documents in reviews
The primary purpose of having a “right sized” model is to facilitate the verification and validation processes applied to the model. To that end, anything that makes the review of the model easier or more complete should be pursued.
- Aim for 30~ 45 minutes: An online review of a model (or portion of a model) should take between 30 and 45 minutes.(9)
- Provide automation results: The reviewers should never have to check that the interface, style, or test requirements are met.
- Define the acceptance criteria: What are you trying to get out of the review and what do you need to provide to get it?
Lean mean, computational machine
So now that we have our “right sized” model, what happens?
- Faster design cycles
- Fewer bugs
- Simplified revision / maintenance
- Higher test / requirement coverage
- Possible increase in required memory
- Possible increase in execution time
In a previous blog I wrote about how to measure the size of a model. Today we focused on what we do with that information; e.g. setting the “right size” which is going to be a function of
- The complexity of what you are developing
- The experience level of your development team
- The maturity level of your development environment.
Highly complex systems will push you toward smaller models while experienced teams and mature environments will support larger models. In the end this will be a balancing act.
- When I was growing up, the main hypothesis was that mega fauna extinction events were tied to large scale natural disasters. I was going to write this blog post suggesting that the death of large models could be attributed to the natural disaster of the COVID crisis. Since this is not the case, I thought I would share this information and take a different tack.
- Even with the section title including “large model” I want to move away from the idea of “large” and “small” models; rather I want to talk about the idea of a “good fit”
- For instance, the sentence you just read. The original draft of it was 2 times as long to say the same thing. It is easy to explain things in 1,000 words, but getting the same thing across in 100 can be a pleasurable feat.
- Informally I would estimate 5~10% of all models is “bloat”; e.g. functionality that could be eliminated through revision.
- It’s unfortunate I needed to write the word “rarely” there. I wanted to write never but I must bow to the fact that there are some instances when the algorithm can only be realized in a way that is unclear to most people. Note as a best practice when this happens, the portion of the code should be extracted into its own function so it can be treated as a black box for the end user.
- The observant among you will notice that I chose “right-sized” to denote the active nature of the work.
- We will cover how to have “informal” meetings in later posts, but the “quick questions” that developers use to break through problems are more difficult when working remotely.
- This is an overdue trend; the ease of “going to the X to test” means that people often will do that instead of paying the up-front cost of developing the Simulation based test environment. In the medium run, the SBTE pays back on the investment.
- In longer online meetings, it is hard for people to remain focused on the topic; without the “in person” awareness of this, it can be hard to maintain the correct tempo of the review.