On Sat, Jul 6, 2019 at 11:21 AM <xxxxxx@ancell-ent.com> wrote:

And 'no-transaction is intended to be something of a lie, in that for databases that support transactions, the single procedure using it as its transaction handle must execute it in a transaction.  The idea is to give the library code the information it needs to execute a single operation with the highest efficiency and safety.  And I'm pretty sure we don't want 'no-transaction to skip triggers.

I think it's better to call it 'auto-commit' for that reason, and to represent it by passing a database instead of a transaction to the various operators.  The effect is as if the transaction is begun, executed, and committed all at once.  How many network round-trips this takes is completely dependent on the implementation, of course.

 

So maybe we want to change it to 'single-transaction, with the caveats that there will be none if the OKVS doesn't support them, and multiple if it calls triggers.

Given the above sketch, it is possible to provide it whether the OKVS does or not -- at a cost, of course, but this is Lisp, where we know the value of everything and the cost of nothing.

 

For maintenance purposes, we need a way to skip trigger procedures, but allow transactions. 

I think constraints are a better name than triggers even on the server side, particularly if they are going to be pure and functional (as I think they really must, for safety's sake). I still really don't know how this should work  I think they have to be done in Scheme and most probably on the client side in the library that provides this SRFI.  A constraint-registering procedure should accept a prefix and a procedure (which gets the actual key, the value, and an insert/delete flag) and returns a pass/fail indication.  An open question is whether failure should be a boolean or an exception (in which case passing means returning an unspecified value).

But the proper scope in time and space of a registered constraint (global, per-machine, per-process, per-thread, per-connection, per-transaction) is still a complete mystery to me.

  the simplest concepts like data types, where SQLite3 isn't even conceptually similar to the basic SQL standard which DB2 UDB, Oracle, and PostgreSQL implement,

I'd say it is.  SQLite columns are dynamically typed, but that's not such a profound difference, especially as most people don't actually exploit it.

 

It's not in Scheme's style to outlaw something like nested transactions.  Using the current API for nested transactions, so that the transaction handles at the user level are unrelated, so threading them like I suggest above is not required, is much better for development and maintenance.

I agree.  I think that if there is no support for nested transactions, okvs-begin-transaction and okvs-in-transaction should raise a continuable exception.  If continued, the inner transaction body is lifted to the outer transaction; otherwise the excepion catcher can do a non-local exit to recover however it thinks best.

That allows two approaches for deciding which triggers to call: either the one that matches the largest fraction of the key prefix, or all that match some fraction of the prefix, starting from the greatest to the least matches, where if a no prefix trigger exists it would be called for every okvs-set!, okvs-delete! or okvs-range-remove! mutation operation (for okvs-range-remove!, for each key-value pair deleted).

How about extending the interface above so that a registered constraint procedure gets a continuation argument which it tail-calls to let higher-level constraints decide?  Best of both worlds and very Schemey.

Does a hierarchy of prefixes make sense

I'd say it makes every kind of sense.  Consider group-of-tables, table, and column specified with longer and longer prefixes.

 

Following the SQL model, if we allow multiple triggers per operation, we could have before and after triggers.

I prefer to have only "before" constraints and leave cascades (which are *not* functional) to some higher-level mechanism outside this SRFI.  You can do a great deal without cascades, especially if you use Scheme prograrmmers, who are expect not to shoot themselves in the foot (much).

 

 And multiples for a prefix, which fire in alphabetical order by the name of the trigger.

Ugh.  In Scheme nothing should depend on things having top-level names (though labeled edges are fine).

 

  If only one, an obvious simple implementation is handing the stored procedure a procedure that includes a transaction as an argument and performs the raw mutation, which it would be called at the desired point.

The difficulty there is figuring out when you are in the constraint procedure and when you are not, which would be needed to avoid recursive invocation of the constraint procedure.

 

We need to make this non-local implicit behavior discoverable, with something like (okvs-trigger-print 'set! prefix) that prints the procedure or procedures that get called; if multiple, in order along with when and where the mutation is done.

If it really must be non-local, then yes.

 

Procedures stored in some place are the only option we have for validation etc., since OKVSes know essentially nothing about the data they store, which is why I'm using the name trigger in alignment with SQL.  We can suggest security measures like running the procedures in a SRFI-172 restricted sandbox, locking down the database if you're storing them in one, not blindly accepting pull requests if in library code etc., but realistically, how much more dangerous is this than letting anyone import and call okvs-delete! etc...?

If we want to lock all this down, we'd need to add something like the SQL security system [...]

Way, way over the top for this SRFI, I'd say.