World's simplest Scheme interface to sqlite3 John Cowan (29 Aug 2020 03:00 UTC)
(missing)
(missing)
Fwd: World's simplest Scheme interface to sqlite3 John Cowan (29 Aug 2020 17:57 UTC)
Re: Fwd: World's simplest Scheme interface to sqlite3 Lassi Kortela (29 Aug 2020 18:44 UTC)
Re: Fwd: World's simplest Scheme interface to sqlite3 John Cowan (29 Aug 2020 22:37 UTC)
SQL API sketch with some code Lassi Kortela (30 Aug 2020 13:24 UTC)
Re: SQL API sketch with some code Lassi Kortela (30 Aug 2020 13:46 UTC)
Re: SQL API sketch with some code John Cowan (30 Aug 2020 20:47 UTC)
Re: SQL API sketch with some code Lassi Kortela (31 Aug 2020 05:02 UTC)
Re: SQL API sketch with some code Lassi Kortela (31 Aug 2020 05:14 UTC)
Re: SQL API sketch with some code John Cowan (31 Aug 2020 15:38 UTC)
Re: SQL API sketch with some code Lassi Kortela (31 Aug 2020 15:54 UTC)
Re: SQL API sketch with some code John Cowan (31 Aug 2020 17:12 UTC)
Re: SQL API sketch with some code Lassi Kortela (31 Aug 2020 19:20 UTC)
Reflection on the database schema Lassi Kortela (30 Aug 2020 13:35 UTC)
Re: Reflection on the database schema John Cowan (30 Aug 2020 19:51 UTC)
Re: Fwd: World's simplest Scheme interface to sqlite3 Alaric Snell-Pym (31 Aug 2020 22:03 UTC)
Re: Fwd: World's simplest Scheme interface to sqlite3 Lassi Kortela (02 Sep 2020 08:38 UTC)
Re: Fwd: World's simplest Scheme interface to sqlite3 John Cowan (02 Sep 2020 17:09 UTC)
SQL statement caching Lassi Kortela (02 Sep 2020 17:21 UTC)
Re: SQL statement caching John Cowan (02 Sep 2020 18:13 UTC)
Re: SQL statement caching Lassi Kortela (02 Sep 2020 18:53 UTC)
Re: World's simplest Scheme interface to sqlite3 Lassi Kortela (29 Aug 2020 11:40 UTC)
Re: World's simplest Scheme interface to sqlite3 Ivan Raikov (29 Aug 2020 04:06 UTC)
Re: World's simplest Scheme interface to sqlite3 Lassi Kortela (29 Aug 2020 11:24 UTC)
Re: World's simplest Scheme interface to sqlite3 John Cowan (29 Aug 2020 17:47 UTC)
Re: World's simplest Scheme interface to sqlite3 Lassi Kortela (29 Aug 2020 18:23 UTC)
Re: World's simplest Scheme interface to sqlite3 John Cowan (29 Aug 2020 22:15 UTC)
Re: World's simplest Scheme interface to sqlite3 Amirouche Boubekki (29 Aug 2020 07:33 UTC)
Re: World's simplest Scheme interface to sqlite3 Lassi Kortela (29 Aug 2020 11:37 UTC)

Re: SQL statement caching Lassi Kortela 02 Sep 2020 18:53 UTC

>     So a table where SQL strings are weak references pointing to
>     sqlite_stmt foreign objects?
>
> They aren't actually weak references, but soft references: that is, they
> break when there is too much memory pressure as opposed to when there
> are no other references.  In this case "memory pressure" means within
> the cache, not general memory.  I think that weak-reference behavior
> (i.e. removing the string when it is GCed on the Scheme side) is not
> really necessary.

True. Weak references might actually undermine the caching behavior if
the GC is too aggressive, since it would reclaim statements right away
if they are not constantly in use?

We also need to be careful to call C destructors on the stmt objects.
Caching is not fun.

>     And it's probably good enough to compare the strings
>     using eq?.
>
> Depends where it is.  If it's on the C side, you don't want to do that
> because gc, hence the idea of a string hash.  If it's on the Scheme side
> eq? is probably good enough.

Scheme-side is best so the FFI doesn't have to unnecessarily convert
Scheme strings to C strings which are not used.

>     If you want to optimize for speed, then (sql-do `("mumble @a @b @c
>     mumble" a ,a b ,b c ,c)) is a bit wasteful as well. SQLite uses
>     positional parameters internally, and the user must convert named
>     parameters to positional ones using sqlite3_bind_parameter_index().
>
> I can't believe the additional cost is worth the loss of clarity and
> readability.

I wonder whether a cache is worth the effort to begin with. Profile
before optimizing and all that. If many wrappers have one, perhaps it
is. There's a school of thought where people are like "always compile
your regexps" but you have to eval them a thousand times for it to
matter. For most purposes, the nicest thing about pre-compiling stuff is
that some errors are caught earlier.

> It's true that PostgreSQL doesn't support named parameters
> in binding, but worst-case we can parse them out ourselves and replace
> them with the $nn format.

Oh damn, Postgres of all databases. "Et tu, Brute?" This is not going to
be fun. Check out how the Python wrapper does it:
<https://github.com/psycopg/psycopg3/blob/7a3e8bf3807d333b06f50a1ecff754a03ed417b2/psycopg3/psycopg3/utils/queries.py#L101>.
I suggest we don't do this, and leave Postgres named parameters to a DSL.

> One nice thing about @ is that it can be
> replaced pretty uniformly except within strings and "-quoted
> identifiers, as it has no meaning in any SQL dialect (which is not true
> of :).

Many people have borne such optimism. Unfortunately I've had it beaten
out of me enough times in the past to be opposed to any escaping that
isn't absolutely required. We should admit that no-one can understand
string escaping, especially against an incompletely known set of
evolving tools, and there's no reward for trying to understand it.

A SQL DSL, on the other hand, would be a very nice thing to have. Since
it has full control over its own output, it can also do any necessary
escaping more safely by restricting its output to a conservative set of
idioms.