Le sam. 6 juil. 2019 à 20:22, John Cowan <xxxxxx@ccil.org> a écrit :

On Sat, Jul 6, 2019 at 11:21 AM <xxxxxx@ancell-ent.com> wrote:

And 'no-transaction is intended to be something of a lie, in that for databases that support transactions, the single procedure using it as its transaction handle must execute it in a transaction.  The idea is to give the library code the information it needs to execute a single operation with the highest efficiency and safety.  And I'm pretty sure we don't want 'no-transaction to skip triggers.

I think it's better to call it 'auto-commit' for that reason, and to represent it by passing a database instead of a transaction to the various operators.  The effect is as if the transaction is begun, executed, and committed all at once.  How many network round-trips this takes is completely dependent on the implementation, of course.

The current draft allows to pass a database instead of a transaction as argument
the underlying backend must handle it the best it can ie. not start a transaction at
all if possible or start and commit the transaction itself (autocommit).

This must be propagated to nstore.
So maybe we want to change it to 'single-transaction, with the caveats that there will be none if the OKVS doesn't support them, and multiple if it calls triggers.

Given the above sketch, it is possible to provide it whether the OKVS does or not -- at a cost, of course, but this is Lisp, where we know the value of everything and the cost of nothing.
For maintenance purposes, we need a way to skip trigger procedures, but allow transactions. 

It seems I did not post a message about it to the mailling list. I am working on validation
and somekind of triggers but not stored procedure. This will rely on SRFI-173.

Whenever one wants to skip validation implemented with hooks what can call
hook-reset! on the appropriate hooks (depending on how the layer is implemented
it can be on okvs or the layer).

I think constraints are a better name than triggers even on the server side, particularly if they are going to be pure and functional (as I think they really must, for safety's sake). I still really don't know how this should work  I think they have to be done in Scheme and most probably on the client side in the library that provides this SRFI.  A constraint-registering procedure should accept a prefix and a procedure (which gets the actual key, the value, and an insert/delete flag) and returns a pass/fail indication.  An open question is whether failure should be a boolean or an exception (in which case passing means returning an unspecified value).

According to the current draft using hooks, failed constraints must raise an exception.
But the proper scope in time and space of a registered constraint (global, per-machine, per-process, per-thread, per-connection, per-transaction) is still a complete mystery to me.

Hooks are registered per process...

  the simplest concepts like data types, where SQLite3 isn't even conceptually similar to the basic SQL standard which DB2 UDB, Oracle, and PostgreSQL implement,

I'd say it is.  SQLite columns are dynamically typed, but that's not such a profound difference, especially as most people don't actually exploit it.
It's not in Scheme's style to outlaw something like nested transactions.  Using the current API for nested transactions, so that the transaction handles at the user level are unrelated, so threading them like I suggest above is not required, is much better for development and maintenance.

I agree.  I think that if there is no support for nested transactions, okvs-begin-transaction and okvs-in-transaction should raise a continuable exception.  If continued, the inner transaction body is lifted to the outer transaction; otherwise the excepion catcher can do a non-local exit to recover however it thinks best.

I have some trouble with supporting nested transaction because so far only
SQLite LSM extension (which is only alpha quality) support them.

That allows two approaches for deciding which triggers to call: either the one that matches the largest fraction of the key prefix, or all that match some fraction of the prefix, starting from the greatest to the least matches, where if a no prefix trigger exists it would be called for every okvs-set!, okvs-delete! or okvs-range-remove! mutation operation (for okvs-range-remove!, for each key-value pair deleted).

How about extending the interface above so that a registered constraint procedure gets a continuation argument which it tail-calls to let higher-level constraints decide?  Best of both worlds and very Schemey.

OKVS will provide hooks that layers can register to. The validation is done by procedures
provided by the layer but might be executed by okvs depending on the kind of validation
(whole transaction validation vs. single mutation validation). So that anytime, the
the validation procedure knows about the actual schema and doesn't have to unpack
the stored bytevectors.

Does a hierarchy of prefixes make sense

I'd say it makes every kind of sense.  Consider group-of-tables, table, and column specified with longer and longer prefixes.

Yes, hierarchy of prefixes make sense. In a row store for instance you can have the
following prefixes:

news row
news pk
news indices column_one
news indices column_two
Following the SQL model, if we allow multiple triggers per operation, we could have before and after triggers.

So far, I hooks in SRFI-168 nstore can be changed by the user using nstore, so they
can act as database triggers. More on that later.

I prefer to have only "before" constraints and leave cascades (which are *not* functional) to some higher-level mechanism outside this SRFI.  You can do a great deal without cascades, especially if you use Scheme prograrmmers, who are expect not to shoot themselves in the foot (much).
 And multiples for a prefix, which fire in alphabetical order by the name of the trigger.

Ugh.  In Scheme nothing should depend on things having top-level names (though labeled edges are fine).
  If only one, an obvious simple implementation is handing the stored procedure a procedure that includes a transaction as an argument and performs the raw mutation, which it would be called at the desired point.

Again, no stored procedure so far. Mainly because I don't want to rely on eval to execute
procedures. As far as I understand, this could be done by the user of the ovks / nstore.
The difficulty there is figuring out when you are in the constraint procedure and when you are not, which would be needed to avoid recursive invocation of the constraint procedure.

The hooks will execute recursively. If a toplevel hook run do some okvs-set! or okvs-rm! the hook associated with okvs-set! and okvs-rm! will be executed again.
We need to make this non-local implicit behavior discoverable, with something like (okvs-trigger-print 'set! prefix) that prints the procedure or procedures that get called; if multiple, in order along with when and where the mutation is done.

If it really must be non-local, then yes.
Procedures stored in some place are the only option we have for validation etc., since OKVSes know essentially nothing about the data they store, which is why I'm using the name trigger in alignment with SQL.  We can suggest security measures like running the procedures in a SRFI-172 restricted sandbox, locking down the database if you're storing them in one, not blindly accepting pull requests if in library code etc., but realistically, how much more dangerous is this than letting anyone import and call okvs-delete! etc...?

If we want to lock all this down, we'd need to add something like the SQL security system [...]

Way, way over the top for this SRFI, I'd say.

Please start a new topic about stored procedures if you _really_ want this SRFI
to specify something about it.

I am working on a new drafts.