Re: NUL-terminated strings and eof-object-terminated generators
Daphne Preston-Kendal 17 Feb 2022 13:17 UTC
On 9 Dec 2021, at 23:43, John Cowan wrote:
> Of course you know what I'm going to say next. There is simply nothing
> wrong with an inbound signal indicating the end of a generator stream, and
> the eof-object is a good choice for that purpose. The dogma that
> generators need to be able to return any object at all has even less
> foundation in actual experience than the dogma that text strings must
> be allowed to contain NUL. Indeed, R7RS-small allows strings to
> contain any Unicode or even non-Unicode characters, but requires that every
> ASCII character be permitted in strings -- with one exception, which I
> proposed and the committee voted for.
I both think that eof-object as a terminator of generators is a bad idea, and that R7RS Large will have failed in its basic goal (to ‘address the practical needs of mainstream software development’) if it does not tighten up what R7RS small says about nulls in strings.
With strings, Large should either make its mind up, or force implementers to make their minds up, between two options: NULs in strings are allowed; or they are not allowed, but a catchable, distinguishable error is signalled if you try to make one. Silent failure, mysterious truncation, and nasal demons are unacceptable in ‘mainstream software development’, where untrusted input from malicious sources is a daily reality.
With generators, I could possibly accept that an error is signalled if you try to create a generator from a sequence that contains an eof-object, but given that generators are lazy (unlike strings), this seems hard (though not impossible) to guarantee in the general case in a way that safely distinguishes such erroneous eofs from the actual end of the stream. And of course, this does not solve the problem that generators cannot be created from any arbitrary sequence (it merely makes it more visible), nor that generators can only return one value per iteration, etc. etc.
Daphne