William D Clinger <xxxxxx@ccs.neu.edu> schrieb am Do., 13. Juli 2017 um 14:42 Uhr:

I apologize for noticing this only after SRFI 149 has become final.
If the SRFI document is in error, it will be too late to fix it,
but we can at least discuss whether the SRFI document really meant
to say what it says.

And we can discuss which path to proceed is the best for the Scheme community.

(In what follows, let me allow to recapitulate the issue for those who will want to join the discussion.)

In the first draft of this SRFI the need for making a choice, namely which ellipses replicate and which iterate, wasn't spelled out explicitly (although a choice was made, see below). One of Al Petrofsky's points was that this SRFI should explicitly spell out that without any choice the semantics would be ambiguous.

If the pattern (a ...) matches (1 2 3), the question I considered for this SRFI 149 was whether a (partial) template ((a ...) ...) should expand into ((1 2 3) (1 2 3) ...) or into ((1 1 ...) (2 2 ...) (3 3 ...)).

Al Petrofky's foo2 example works with the former semantics, but not with the latter semantics.

On 9 March, Al-Pastor Petrofsky posted the following example:

> (define-syntax foo2
> (syntax-rules ()
> ((foo2 (axis ...) (coordinate ...) ...)
> '(((axis coordinate) ...) ...))))
>
> (foo2 (x y) (0 0) (0 3))
> => (((x 0) (y 0))
> ((x 0) (y 3)))
>
> (foo2 (x y) (0 0) (0 3) (3 4))
> => (((x 0) (y 0))
> ((x 0) (y 3))
> ((x 3) (y 4)))

As Will Clinger writes, Larceny has been supporting the former semantics for years (I have to apologize that I didn't run any tests on Larceny when I wrote this SRFI). On the other hand, Chibi, Kawa and Sagittarius support the latter semantics.

The foo2 example gives us an application for the former semantics. The latter semantics, however, has also valid use cases (that wouldn't work with the former semantics):

(define-syntax gen-extract

(syntax-rules ()

((gen-extract ((x ...) generator) . body)

(let* ((x (generator)) ...) . body))))

This macro is an ordinary R7RS-macro. For example,

(gen-extract ((x y) g)
(list x y))

expands into

(let* ((x (g)) (y (g)))

(list x y))

We may want to extend the macro so that it supports extracting the values of more than one generator (compare to one-binding let vs let*):

(define-syntax gen-extract*

(syntax-rules ()

((gen-extract* (((x ...) generator) ...) . body)

(let* ((x (generator)) ... ...) . body))))

The hypothetical macro writer expects that

(gen-extract*

(((x y) g)

((p q) h))

(list x y p q))

expands into

(let*

((x (g))

(y (g))

(p (h))

(q (h)))

(list x y p q))

It does so with the latter semantics, but not with the former semantics (e.g. it works with Chibi (apart from a problem with the double ellipses in the template), Kawa, Sagittarius).

This SRFI made a choice between the two semantics. The choice it made was in favor of the latter semantics (that is making the gen-extract* example work, not the foo2 example). It is spelled out in the SRFI as follows: "If a pattern variable in a subtemplate is followed by more instances of <ellipsis> than the subpattern in which it occurs is followed, the input elements by which it is replaced in the output are repeated for the innermost excess instances of <ellipsis>". In fact, this wording was suggested by Al Petrofsky in his post on this mailing list. In the gen-extract* example, the innermost occurrence of <ellipsis> in the template replicate a particular instance of (generator), while the outer occurrence iterates over the instances of (generator).

SRFI 149 does not cite the gen-extract* macro to justify its choice (such a justification would have been arbitrary, anyway, because this SRFI could have cited the foo2 macro to justify the other choice). However, it gives an abstract reason in the rationale why the choice made by SRFI 149 was not an arbitrary one but one founded on theoretical considerations, namely functoriality:

If ((_ . <pattern>) <template>) is a valid syntax rule of the syntax-rules-pattern language, the "lifted" rule ((_ <pattern> ...) (<template> ...)) becomes a valid and meaningful syntax-rule with the semantics of this SRFI.

So, if we view a syntax rule as a function from, say, widgets to gadgets, I can easily lift this syntax rule to a function from list of widgets to list of gadgets with the semantics of the SRFI 149, that is the latter semantics from above.

If SRFI 149 was intended to describe a semantics in which the foo2
example works as expected by Petrofsky and myself, then its

(let-syntax
((foo
(syntax-rules ()
((foo (a b ...) ...) '(((a b) ...) ...)))))
(foo (bar 1 2) (baz 3 4)))

example should evaluate to (((bar 1) (baz 2)) ((bar 3) (baz 4))),
as it has been doing in Larceny for several years, not to the
result stated in SRFI 149.

The result stated in SRFI 149 conforms to to choice SRFI 149 made when choosing the latter semantics.

So much for reviewing. As for options for the community to proceed, I do see the following:

(1) Scheme implementers do not agree on one semantics, e.g. Larceny sticks to its semantics; Chibi, Kawa and Sagittarius stick to SRFI 149's semantics.

This is probably the worst outcome because it would mean that one can neither write a portable "desirable" macro like foo2 nor gen-extract*. (Of course, it is possible to write macros with the same desired effect as foo2 and gen-extract* using only the R7RS, but not as easy.)

As in that case, SRFI 149 won't find general consensus, a reduced version of SRFI 149 will have to be written that is only concerned with direct repetition of ellipses in templates (this part of SRFI 149 seems uncontroversial).

(2) Larceny abandons its semantics and joins the SRFI 149 camp. There is the risk of existing code that would break but such code can't be portable Scheme code.

(3) The community agrees on that SRFI 149 made a suboptimal choice and that the former choice of semantics from above is a better one. In that case, a revision of SRFI 149 will have to be written. An argument against SRFI 149's choice should include a justification why the other choice is more naturally despite of SRFI 149's arguing on theoretical grounds and functoriality.

(4) In fact, both macros foo2 and gen-extract* work with Larceny (apart from the fact that Larceny does not allow ellipses in a row in a template), so Larceny neither implements the former, nor the latter semantics, but some middle ground. Thus, Larceny's current semantics look superior to the choice made in this SRFI and to the other choice discussed above.

There is one outermost ellipsis in the pattern of gen-extract*, following a subpattern containing both pattern variables x and generator, while there is no such outermost ellipsis in the pattern of foo2 that follows a subpattern containing both pattern variables axis and coordinate. For functoriality, only outermost ellipses as in gen-extract* matter.

In gen-extract*, the patterns (x ...) and generator have a common ellipsis following them so it makes sense that generator is iterated whenever x is iterated twice. On the other hand, in foo2, the patterns axis and (coordinate ...) do not share a common ellipsis following them both, so it makes sense to choose a different ways for template expansion here, which Larceny does.

I have not thought of a formal specification for that semantics. Maybe Will Clinger can describe the semantics implemented by Larceny in a few sentences.

If Larceny's semantics can be formalized and there is agreement on them, a new SRFI should be written.

(5) One could think of syntactical means to distinguish between the semantics proposed by this SRFI and the semantics of Larceny. E.g. the foo2 example could become

(define-syntax foo2
(syntax-rules ()
((foo2 (axis ... ...) (coordinate ...) ...)
'(((axis coordinate) ...) ...))))

The extra ellipsis after axis here should mean that replication in the template shall be moved one ellipsis to the left when compared with the bare SRFI 149 semantics.

Such a syntactic extension could also be discussed here before a new SRFI is written.

These are my thoughts for the moment. I think that path (4) looks most promising. If this what Al Petrofsky's wanted to imply in his post, I apologize for having not realized it.

Marc