I have no idea how to achieve this.
I create this new topic to raise awareness.
Date: Saturday, June 29, 2019 9:46 AM
That is very difficult to do validation at this level. OKVS manipulates bytevectors, to validate keys and values it would require to unpack the bytevector which is not know at this level. I acknowledged that validation is missing piece in the whole framework (SRFI-167 + SRFI-168), I am thinking about ways to improve the situation, but that will not be part of any current SRFI.
A simple approach would be to associate a simple Scheme function with each open database that is called before inserting any key-value pair. It is passed the key and value as its arguments, and returns #t if the combination is valid and #f if it is not. Of course the behavior of this function would depend on how the store was used. That would be straightforward to implement within the library.
That's the sort of simple thing I was thinking of, and I think it would be in the scope of either or both SRFIs.
My thought is that validation should be specified in higher level abstraction, but that one should be able to hook it on okvs.
If the constraint is to be persistent, it would need to be stored in the database under some well-known key, which would mess up the complete freedom of the user to use any key desired. If that were overcome, a fully general constraint would have to be stored as a lambda expression and eval'd in some minimal library (which would be worth defining anyway, probably).
That's a capital idea, although evaluating random external code invites mischief. Arguably more beyond the scope of these SRFIs, especially since it would be nice to store multiple constraints, but it would be extremely useful, and help bridge the validation capability gap between simple OKVSes and the better RDBMSes.
RDBMS are row stores and provide shallow and type validation (and sometime foreign key constraints). That is when the user writes a okvs key-value pair, all the data that will allow some kind of validation to be relevant is available:
(okvs-set! db (pack 'news 'row 1) (pack "SRFI-167: OKVS" "Proposal to support Ordered Key-Value Stores in Scheme implementations" 2019))
(okvs-set! db (pack 'news-tag 'row 1) (pack 1 "scheme")) ;; news foreign-key and tag
Whereas the nstore, it would be coded as:
(nstore-add! db '(news-1 title "SRFI-167: OKVS"))
(nstore-add! db '(news-1 body ""Proposal to support Ordered Key-Value Stores in Scheme implementations""))
(nstore-add! db '(news-1 year 2019))
(nstore-add! db '(news-tag-1 news-id news-1))
(nstore-add! db '(news-tag-1 tag "tag"))
In that case, it would be possible to validate the type of individual nstore-add but not the row-equivalent as whole
except IF there was a transaction hook and track with tuple were touched. Mind that the example rely on "auto-commit"
so in that case whole "document" validation will not be possible.
I believe that upfront validation (schema-on-write) is to database programming what static typing
is to programming languages. Respectively schema-on-read is similar to dynamic typing.
In this regard my take on it is that "gradual" or "optional" typing could be a good thing to support
or the very least not make it impossible.
Another thought, postgresql upfront schema validation is in no way a full validation framework,
in the sense that one must still assume garbage in, garbage out. Also, PostgreSQL validation,
is in no way good enough for end-user reporting because the error that is raised has not enough
information even if you parse (!) the error string. That means that validation must still happen in user code.
Yet another thought, validation in the sens of a okvs-set! predicate assume that
if the key-value pair pass the test, the whole database will still make sense, that
is a false promise it makes to the user.
AFAIK and I am far from being a specialist on the subject and my knowledge on it might be old:
Whole schema up-front validation does NOT exist. It always assume if it is locally sound, it should
be globally sound. Which like I said previously is still garbage-in, garbage out in some occasions.
Still I agree that some validation facility will be a good thing to have.