How deep is your data?

I recently had a conversation with a client about how they instantiated constants for their model.  Thier approach was to group common parameters together into a structure.  What I pictured was something like this

In this instance, we have a small structure, one layer deep with 4 elements.  This would be enough information to perform the calculations required to transform a throttle sensor voltage into a throttle position.

However, what they showed me was something quite different.  In their instance, they had a hierarchical structure that was, in some places, 7 layers deep.   The single structure contained not only all the parameters required for a single component but for multiple models.

The deep…

This isn’t the first time I have seen a structure like this, in general, they grow organically.  As multiple people work on a project they want an “easy” way to share data and, at first, when it is small, the method works well.  However, as the structure grows in size several problems start to emerge.

1. Where is my data: the first problem with large data structure is finding the data.  Even the most logically organized structure
2. Waste of space: deep structures inevitably end up with unused data.
3. Repeated data: Going along with the “where is y data” is the “repeated data.  People will add the same data to multiple locations.
4. Customization: with a large data structure you have to configure the whole data structure as one.

Finding balance

The argument against flattening structures was “If we break them up we will have thousands of parameters”.  While that was factually correct it missed the fact that they already had 1000’s of parameters, just in a deep structure.  The advantage of the flat format are

1. Ability to easy find parameters
2. Ability to customize on a parameter by parameter basis
3. Only used parameters are in the generated code

There are some disadvantages, related to how the parameters are stored in files; a single structure can be stored easily in a single structure.  With multiple parameters, a storage format needs to be determined.  Standard approaches include

1. Use of MATLAB files
2. Use of .mat files
3. Use of Simulink Data Dictionary
4. Use of an external data base

Any of these approaches can be used to organize the data.