Marc Nieper-Wißkirchen wrote:

> > It looks to me as though the "c" inserted by line 135 of 147.scm
> > has the same meaning as the "c" in the test (because neither is
> > bound in the context where they are inserted) so free-identifier=?
> > recognizes them as equivalent.
>
> The "c" inserted by line 135 of 147.scm is a pattern variable bound to this
> syntax rule. Wouldn't this have to be renamed to preserve hygiene?

Maybe.  People often talk of renaming and hygiene as though it's all
simple and obvious, but I don't know of any authoritative formal
specification that explains how to deal with macro-defining macros,
syntax-case, et cetera.  (I know of several attempts, including my
own, but they don't entirely agree with each other.)

For example, we have to distinguish between tentative renaming (in
some intermediate form) and ultimate renaming (in the fully expanded
code).  The real question here is which intermediate forms should
match and which should not.

The tracing I did yesterday convinced me that Larceny's macro expander
was renaming too aggressively and/or failing to match often enough,
however you want to look at it.  That is why Larceny was generating
an error with your sample implementation of SRFI 148.  There's no doubt
about that.

Today I pushed a change to the macro expander that reduced its renaming
and/or increased its matching of literals, again however you want to
look at it.  I thought I had brought it in line with the R6RS specs,
but your example shows my changes went too far:

> I tried the following code at the repl:
>
> (define-syntax foo
>   (syntax-rules ()
>     ((foo bar x)
>      (define-syntax bar
>        (syntax-rules (x)
>          ((bar c) 'matched)
>          ((bar z) 'unmatched))))))
>
> (foo bar c)
> (bar q)
>
> Chez, Chibi, Racket (R6RS), Larceny (R7R6, version from yesterday) all
> evaluate the last expression to matched.
>
> This seems to indicate that all systems hygienically rename the pattern
> variable c in the output of the foo macro.
>
> This seems to contradict your recent experiments (e.g. (define c #f)) with
> the 147.scm code. Did Larceny change its behaviour between yesterday and
> today with respect to the code snippet above?

Yes.  Larceny needs to behave the same as those other systems on simple
examples such as the one above, so I haven't yet found the right compromise
between the overly aggressive renaming Larceny's been using for ten years
and the less aggressive renaming that made your sample implementation work.

> >  From R6RS 11.19:
> >
> >     A literal identifier matches an input subform if and only
> >     if the input subform is an identifier and either both its
> >     occurrence in the input expression and its occurrence in
> >     the list of literals have the same lexical binding, or the
> >     two identifiers have the same name and both have no lexical
> >     binding.
> >
> > R7RS 4.3.2:
> >
> >     An element in the input matches a literal identifier if and
> >     only if it is an identifier and either both its occurrence
> >     in the macro expression and its occurrence in the macro
> >     definition have the same lexical binding, or the two
> >     identifiers are the same and both have no lexical binding.
> >
> > The R6RS is more precise here, with the "have the same name"
> > explaining what the R7RS leaves open to more imaginative
> > interpretations by saying "are the same".
>
> Renamed identifiers are certainly not the same (e.g. are not
> free-identifier=?); by the very meaning of "rename", they also do not have
> the same name.

That's an overly simplistic view.  Tentative renaming of identifiers
(coloring, marking, whatever you want to call it) does not prevent
them from being matched by free-identifier=?, whose semantics is
described by R6RS 11.19 and even more explicitly by the R6RS Standard
Libraries document, section 12.5:

    The free-identifier=? procedure returns #t if and only if
    the two identifiers would resolve to the same binding if
    both were to appear in the output of a transformer outside
    of any bindings inserted by the transformer. (If neither
    of two like-named identifiers resolves to a binding, i.e.,
    both are unbound, they are considered to resolve to the same
    binding.) Operationally, two identifiers are considered
    equivalent by free-identifier=? if and only [if] the topmost
    matching substitution for each maps to the same binding
    (section 12.1) or the identifiers have the same name and
    no matching substitution.

The two R6RS documents are pretty clear about this.  Not so clear
is whether and when an identifier inserted via macro expansion is
to be considered bound (for the purposes of the paragraphs quoted)
because it ends up as a pattern variable in a use of syntax-rules
created by macro expansion.  That is subtle, because the question
of whether the inserted identifier becomes a pattern variable or
literal has to be determined by matching it against the literals,
which means you have to do some form of matching before you can do
the matching described for free-identifier=?.  This is probably
where I went wrong.

(In Larceny, syntax-rules is actually a macro, not core syntax, so
I have to think about these issues in terms of syntax-case, not
syntax-rules.  That's one reason I tend to discuss these matters
in terms of free-identifier=? and the like.)

> I still do not understand why the apparent renaming of "c" in the 147.scm
> does not work. (Or rather, why Chibi and Sagittarius work, but Larceny
> doesn't, while Larceny passes the above test.)

Today's version of Larceny, which is the one that's able to run
your sample implementation of SRFI 148, does not pass the above
test.

> Of course it may be that I miss the forest for the trees and that I don't
> see the obvious.

The high-level explanation for what's going on is that yesterday's
version of Larceny's macro expander was overly hygienic, while
today's version is insufficiently hygienic.

For that to make sense, you have to realize the word "hygienic"
is somewhat ill-defined in these corner cases.  It would be nice
if consistency between implementations were achieved entirely by
following a mathematically precise specification, but the truth
is that we end up tinkering with our systems until they pass the
test cases people care about.

Will