Email list hosting service & mailing list manager

Systems lie about persistence and consistency promises. A lot. hga@xxxxxx (10 Jul 2019 22:10 UTC)
SQLite only SRFI? hga@xxxxxx (11 Jul 2019 01:49 UTC)
Re: SQLite only SRFI? John Cowan (11 Jul 2019 03:23 UTC)
Re: SQLite only SRFI? hga@xxxxxx (11 Jul 2019 08:19 UTC)
Re: SQLite only SRFI? John Cowan (11 Jul 2019 13:32 UTC)
Re: Systems lie about persistence and consistency promises. A lot. Arthur A. Gleckler (10 Jul 2019 22:51 UTC)
Schemepersist, anyone? hga@xxxxxx (10 Jul 2019 23:40 UTC)

Systems lie about persistence and consistency promises. A lot. hga@xxxxxx 10 Jul 2019 22:10 UTC

> From: Lassi Kortela <xxxxxx@lassi.io>
> Subject: Re: scsh-0.6.7 -> scsh-0.7 has substantial changes, and introduces grievous error
> Date: Wednesday, July 10, 2019 3:32 PM

[ The troika of John, Lassi and Harold excise sync-file (born broken,
  and still so on Linux) and sync-file-system (badly named, sort of
  broken by design but traditional, and broken in implementation. ]

> And there was much rejoicing :)

[...]

> It could someday be the remit of a SRFI or library.

> Not sure if this is what you mean, but disk syncing is definitely
> enough of a black art that it could merit its own SRFI that contains
> nothing but a sync procedure (and a lengthy Rationale section full
> of caveats about it :)

That was an offhand thought, but thinking about it more, Scheme's
strategy could be something like this:

Continue crossing out fingers for now, knowing that manual recovery
of some sort may be needed *anytime* you attempt to write to "disk".

Depend on subject domain experts like the SQLite team and their whole
software stack when you need the best guarantees available today.  Ah,
along with notes about system configuration details and trade-offs.

Potentially develop a Transaction Manager SRFI that breaks absolutely
no ground in low level implementation details, concept, and primitives
and code we ... borrow from the best.  But is easy to grok and use.

It needs to be at a higher level than primitives like fsync because
correctness today is inextricably tied to your application, in how it
restarts, checks for errors, and recovers what it can.

This has been a interest of mine since I was given a job to make a
custom database that just stored data reliable; I'll put it on my
long term TODO list, during/after tackling the SQL RDBMS issue.

- Harold