How can we handle as many quirks as possible in general fashions? hga@xxxxxx (18 Sep 2019 01:32 UTC)

How can we handle as many quirks as possible in general fashions? hga@xxxxxx 18 Sep 2019 01:31 UTC

An issue which has the potential to drive us crazy, and perhaps has
something to do with Scheme DBI/DBD style stacks not being
historically maintained outside of Gauche, is that every RDBMS has
it's quirks.  Plus I'd like to sweep in non-RDBMSes to the extent
that makes sense.

While there will of course be an execute-literal string procedure that
can return results, as well as the DBI level of the API not knowing
the query language at all I think, how should we pass though
non-default data type conversion directives, and what database specific
extensions and quirks don't fit in this paradigm, like callbacks?

Previously mentioned were MySQL string columns being allowed to have
different encodings.  Which actually sounds like a useful feature in
our polygot, long time if ever before everyone uses Unicode for
everything.  It's unrealistic to provide first class support for
every databases' quirks, although if we take our time, and work with
a large variety of databases, perhaps....  Anyway:

> From: Alaric Snell-Pym <xxxxxx@snell-pym.org.uk>
> Subject: Re: Binary S-expressions
> Date: Tuesday, September 17, 2019 10:52 AM
>
> On 17/09/2019 14:48, Lassi Kortela wrote:

> [ On the the encoding between Scheme types and database values... ]

> As for strings... hopefully, the driver will negotiate with the database
> to find out what encoding is expected, but IIRC with MySQL this can vary
> between columns in the same database so I'm not sure if there's a nice
> way to automatically use the correct encoding in all cases:
>
> https://dev.mysql.com/doc/refman/5.7/en/charset-column.html
>
> The best we can do there is probably to have a "default encoding" set as
> part of the options when making a connection, maybe with the option to
> override it per-column when binding strings into queries through some
> MySQL-specific mechanism....

Per column optional pass throughs ... could that be a general mechanism
to possibly accomplish other things?  How else are we going to say, *this*
value gets interpreted in *this* non-default way?  How ugly might that
make the API?  Here's a bit more from Alaric on this:

> However, other users of the same database will often fail to send
> correctly encoded text, due to not realising that it's a problem and
> just assuming that whatever string of bytes their app has will be
> interpreted the same by the database [messy recent real world
> example].  For normal users, we should magically do the right thing
> to make their application correctly send strings to and from the
> database; but we should also cater to "advanced" users having to
> suffer that kind of work, if we must, perhaps by having some
> mechanism to request that strings be returned from the DB as
> bytevectors for them to do their own decoding on...

- Harold