Re: SRFI 207: String-notated bytevectors Lassi Kortela 18 Aug 2020 06:51 UTC

>     I think the main reason is that if we allowed the full Unicode range
>     and specified UTF-8 encoding, sequences like \x80; would be ambiguous.
>     Either the byte 80 is meant or the bytes corresponding to the UTF-8
>     encoding of U+0080.
>
> I agree.  With this regard, I think the syntax Alex suggested earlier
> seems to work well.  This way, octed sequence that's not valid as utf-8
> can be included without ambiguity:
>
> #u8("abcde" #x80 "efghi")
>
> In the string part, we can say either ASCII-only, or utf-8 encoded string.

That syntax also had the advantage that the R7RS #u8(...) vs R6RS
#vu8(...) discrepancy is dodged. Implementations would allow strings
inside whichever # prefix they currently have. In case they support both
prefixes, allow it in both.

> In order to allow implementations to extend #u8"..." so that "..." can
> be any string allowed by the implementation (*), I want to suggest to
> rename the sequence "\xHH;" of this SRFI into something different like
> "\yHH;". I think this is a good thing because "\xHH;" in strings and
> characters really means something different for bytes greater than
> #x7F as soon as we encode it in UTF-8 (which string->utf8 does).

I'd vote to support fewer kinds of characters in bytestring literals to
avoid confusion. Here the #u8("foo") syntax would have the advantage
that we'd have a good excuse to drop \x; escapes completely :) The
bytevector #u8("Hello" #x20 "world" #x0a) would be equivalent to
u8"Hello\x20;world\x0a;" in the current draft without using any escapes.

If \xHH; is not supported, confusion about its meaning and permitted
range of characters is also avoided.