Re: Mechanism vs policy Lassi Kortela 26 Sep 2019 07:40 UTC

> While protocol buffers have their faults, this is one of the things that
> their designers worked hard at, and which I admire.  Protocol buffers can
> contain fields that my program doesn't recognize, but my program can
> manipulate the parts that it understands and pass those and the rest along
> to another program that might understand different parts.  For example, I
> can write a program that doesn't understand anything at all about the
> protocol buffers it is receiving, but can store them on disk and retrieve
> them when a program that does understand them requests them.

Indeed. This is beating a dead horse a bit but I feel I didn't express
my reasoning clearly enough in the last mail. My main point was that we
can't always control what data we need to get at. The poorly-structured
data using questionable datatypes can be other people's data. Or it can
be a merging of several other people's data from several sources, or a
Lisp program that forces us to use syntax or data types that are perhaps
too rich for the job. Often enough, the poor data may be my own data
that I jotted down when I was tired and couldn't bother to clean it up.

So if we want to get at that data, we have two choices: 1) either stash
it into an all-encompassing format that can easily ship it around, or 2)
keep it in its current format, or amalgamation of different formats,
shipping each part around separately, keeping in mind the limitations of
each of those formats, and taking the time to clean up the data before
storing it in the final format.

The above finally expresses my concern to my satisfaction: If the final
format and the intermediate format can be the same one, that reduces
cognitive load a lot for these tedious clean-up-and-merge-data tasks.
And the only way to accomplish that is if the format can represent every
data type with no range limits. I think John's original idea about the
ASN.1 super-format was based on a similar idea. I'd just like a textual
1:1 equivalent too, again to reduce cognitive load: text vs binary is
often a constant question in the back of the mind while developing a
project. If we can eliminate that unpleasant mental diversion, that's
again a big reduction in cognitive load on a matter that is ultimately
unimportant to the task at hand.