OKVS is an abstraction that is *loosely* based on FoundationDB (FDB).
That is, it does not expose *all* FDB features, for maximum
performance in a very large database, roughly more than 1TB, you would
be better served by relying on FDB bindings directly. Mind the fact,
FDB works fine in standalone mode, it does also have an in-memory
backend.

The big difference between FDB and the embedded OKVS (RocksDB,
WiredTiger, SQLite LSM extension) is that the latter rely on cursors
with procedures like cursor-seek, cursor-previous, cursor-next, and
cursor-close! In those embedded OKVS, it is trivial to implement
leaderboards, sorted mapping or ranked set. In FDB, you can workaround
it using engine-ref* (see below) and by implementing your own
datastructure (see for example
https://github.com/FoundationDB/fdb-record-layer/blob/master/fdb-extensions/src/main/java/com/apple/foundationdb/async/RankedSet.java).

That is, one could specify a cursor-based OKVS that will be a
complement to SRFI-167.

I choose to base OKVS on a subset FDB interface, because there is less
procedures. That will hopefully, help spread the idea that the ordered
key-value stores are a better abstraction for persistent on-disk
storage with strong guarantees compared to, so called, RDBMS.

Here is a few notes after working more with those two SRFIs:

- SRFI-168 nstore is good as a generic store. That is when you want to
prototype quickly without any optimizations. The only procedures that
I can see that can be good to have is a nstore->list procedure and
list->nstore.

- Related to versioned nstore (formerly fstore), it needs some love in
particular I put too much features in that abstraction e.g. the
snapshot should be in a higher level abstraction. [0]

[0] https://git.sr.ht/~amz3/chez-scheme-arew/tree/master/src/arew/data/base/fstore.scm

Regarding SRFI-167:

- Regarding the engine's invoker procedure pass arguments as rest
arguments and does not check the number of argument. This can be
improved. That procedure is private and not specified.

- It will be very handy to have alist->okvs and okvs->alist.

- Taking inspiration from FDB, (engine-ref* engine okvs spec) could be
added. SPEC can be a key, a selector or a pair of key or selector. A
selector is a key associated with an offset and an equality flag.
Similarly engine-remove* could be added. That covers several use cases
and can avoid the need to (re) implement a ranked set on top of okvs.
See https://apple.github.io/foundationdb/api-python.html#key-selectors

- engine-prefix-range-remove! could be added, this is sugar syntax
over engine-range-remove! Since the procedure strinc is not part of
the public interface, it would make sense.

Regarding engine-pack and engine-unpack that are in the end
lexicographic packing and unpacking:

- The sample implementation does not support nested lists, this is
trivial to add, which in turns makes handling subspace prefixes much
more easy (e.g. in nstore),

- I think it requires to support pairs instead of lists

- It might also support vector type

- I think it would be better if engine-pack was taking its argument as
a list instead of a rest argument. That will, I think, avoid a copy
when doing something like (apply engine-pack engine args). That way,
taking inspiration from FDB, it could take a raw bytevector prefix as
optional second argument. Also, it will make it more symmetric with
unpack that returns a list.

Thanks a lot for the help.

--
Amirouche ~ https://hyper.dev