tl;dr: they can be perfected, but I need more time for another SRFI.
Can we still withdraw the SRFI so that people do not lose time trying
to implement those ?

According to SRFI rules:

> A final SRFI may later be withdrawn, but only if it has been replaced with a newer SRFI and the author of the original SRFI agrees. In that case, there will be a "See also" link from the withdrawn SRFI to the new one. Note that withdrawn SRFIs are still present on the web site, in full — just marked withdrawn.

So, I would need a new SRFI to replace them. That is somewhat a
problem, because I think it will mislead people into thinking that it
is perfect which is not the case. Tho, it is useful. I made nomonufu
and guile-babelia that are still available.

Disclaimer: I did a lot of work before submitting those SRFI, but I
underestimated how deep and large the topic is, even after
FoundationDB release and its documentation that was the first document
to describe patterns and coin the term "layer" to describe what I used
to call "abstraction" (today I prefer the term 'extension'). And even
if my proposal might have been misguided by the fact that it was an
opportunity to get feedback on my work, I did, and it is marvellous. I
want to thank Arthur, John, Harold, and Vasilij.

## What wrong with SRFI-167

- The big mistake was to try to support both FoundationDB (FDB) that
is really the only mature distributed okvs, and embedded OKVS such
(sqlite-lsm-ext, rocksdb, wiredtiger), because I see no compelling
reason to support both: FDB support will be weak (no watches, no high
contention allocator and related transaction conflict operations, not
atomic operation, and others such as approximate bytes count, and
maybe in the future: approximate key count, the latter being a very
interesting feature to implement query optimizers), and the other side
the cursor API of embedded okvs makes it easier to implement ranked
set, priority queue and leader board. My current position is along the
line of: small web app or desktop app: go embedded okvs, big project:
go dedicated FoundationDB bindings. SRFI-167 is not good enough for
both.

- Not using generics was a big mistake, the good solution for what is
called the `engine` are generics.

- The specification of lexicographic packing procedure is absent,
again generics or something more fine-tuned for the task at hand will
help the conversion of any scheme data types into bytes in one pass
that would reduce uninteresting code such as bag->alist or
mapping->alist et al.

- Specifying the packing and unpacking in the engine was a mistake.
Each extension might use a different packing strategy, for instance
SRFI-168 (nstore) in my copernic project only needs an "exact match"
comparison.

- `pack` takes a rest argument which is not efficient and rather
original. It should take a prefix as argument and an obj.

- regarding okvs-open can help, but in my preliminary work to replace
it, I started with chibi keywords, the situation is more complicated
and I prefer to use a record type.

- config optional argument is wanna-be compatibility layer with
wiredtiger, and I stopped using wiredtiger because it is too much
memory hungry, and because of the license

- make-default-state was supposed to be used with hooks and possibly
FDB version stamped keys, today I use none of those.

- I replaced okvs-range-remove and okvs-remove with simply
(okvslite-remove handle key [other]) that has less interface surface
and is still easy to understand.

- okvs-range, okvs-prefix-range, and their myriad of arguments are
replaced with (okvslite handle key [other [offset [limit]]]), it still
return a generator, and the reverse requires only to swap KEY and
OTHER.

- hooks may only allow validation or triggers but it was never done,
FDB rejected my PR in Python bindings, also post-commit hook is
required to implemented watches (fdb) or notify (psql)

The early rework draft is available at
https://github.com/scheme-live/live/blob/okvslite-and-co/live/okvslite/README.md#status

## What wrong with SRFI-167

- Lexicographic packing is really powerful, that is what makes it
fractal ie. one can create indices using one of the tuple items or
even a full okvs. But so far, in my use case it is overkill. Given a
(uid, key, value) if one wants to index everything that has key=data,
they might as well use another subspace. So the feature should be
opt-in and make possible the performance gain related to pack scheme
objects using something that is lexicographic.

- In the vnstore, that rely on an nstore where n=5, I can reduce the
index factor from 10 to 6 (according to make-indices), it is almost
half more space available (for cats and dogs pictures) that
optimisation is only possible when the value part of the okvs can set
and retrieved: So, two more procedures are required:

(nstore-set! ... items value)
(nstore-ref! ... items)

Also querying was replaced by a single procedure that makes it
possible to include a query optimizer:

instead of:

(nstore-query (nstore-select ... pattern0) (nstore-where ... pattern1) ...)

It can be simplified such as:

(nstore-query ... (list pattern0 pattern1 ...))

Under the hood the naive implementation is to execute the above query.
The new procedure allows to hide the particular implementation of the
query engine.

What do we do with SRFI-167 and SRFI-168 ?

--
Amirouche ~ https://hyper.dev