Email list hosting service & mailing list manager

Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela (22 Feb 2020 00:48 UTC)
Comparing current and potential future syntaxes Lassi Kortela (22 Feb 2020 01:25 UTC)
Re: Comparing current and potential future syntaxes John Cowan (22 Feb 2020 22:17 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments John Cowan (22 Feb 2020 21:22 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela (22 Feb 2020 22:17 UTC)
Hygiene and allow-other-keys Lassi Kortela (22 Feb 2020 22:39 UTC)
Re: Hygiene and allow-other-keys Shiro Kawai (22 Feb 2020 23:48 UTC)
Re: Hygiene and allow-other-keys John Cowan (22 Feb 2020 23:52 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments John Cowan (22 Feb 2020 23:40 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments Shiro Kawai (22 Feb 2020 23:58 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela (23 Feb 2020 00:12 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments John Cowan (23 Feb 2020 01:03 UTC)
Required keyword arguments Lassi Kortela (23 Feb 2020 07:39 UTC)
Re: Required keyword arguments John Cowan (23 Feb 2020 16:35 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments Marc Nieper-Wißkirchen (23 Feb 2020 08:33 UTC)
Re: Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela (23 Feb 2020 10:09 UTC)

Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela 22 Feb 2020 00:48 UTC

I'm ready to wrap up this SRFI as soon as we decide on what to do.

### Naming

There was wide agreement:
* Rename keyword-lambda to lambda/kw.
* Rename keyword-call to call/kw.

I'll also add define/kw, which is the obvious combination of define and
lambda/kw.

### Performance, aka should define/kw make lambdas or macros?

I designed lambda/kw as making real lambdas (using a rest argument for
the keywords in Scheme implementations that don't have real keyword
args) for simplicity: if keyword lambdas behave like ordinary lambdas in
as many ways as possible, there is the least potential for surprises,
and maximally easy to add keyword arguments to existing Scheme
procedures that don't have any yet.

Marc Nieper-Wißkirchen suggested that define/kw could make macros
instead of lambdas, which would make keyword calls easier to optimize.
That strategy makes me a bit uneasy since making keyword lambdas subtly
different from other lambdas makes it a more complicated abstraction,
with potentially more edge cases that cause trouble to some users.
However, I don't have any particular concrete problems in mind;
simplicity is just a design instinct.

A good point of comparison is when OpenBSD designed the strlcpy() and
strlcat() safer string functions for libc. They could certainly have
better characteristics, but the priority was to design drop-in
replacements for strcpy() and strcat(). I had a similar design mentality
of trying to make lambda/kw a drop-in replacement for lambda.

Common Lisp compilers can optimize keyword calls, but custom compiler
macros may have to be written. Gambit, Kawa and Racket probably have
some optimizations of their own. I'm not really qualified to judge the
issue. Preliminary benchmarks (making lambdas with way too many keyword
arguments) showed predictable slowdown in the portable implementation,
but calls were still reasonably fast. They might not be fast enough in a
tight inner loop; my recommendation would be to avoid fancy coding
techiques such as keyword arguments in the hot path. FWIW, Dybvig is
generally cautious about adding new language features to Chez Scheme
since their performance implications are unclear.

### Syntax

The current syntax is (lambda/kw (a b c (d e f)) ...) and (call/kw 1 2 3
(d 4 e 5 f 6)). It's a non-ideal compromise to allow implementations
using only syntax-rules. We came to the conclusion that almost all
Schemes support another macro system, so we agreed to drop the
syntax-rules compatibility requirement. That gives more design freedom.

Following that decision, this was the favorite lambda syntax:

(lambda/kw (a b c :key d e f) ...)

The precise naming of :key vs key: vs &key is up for debate.

This Common Lisp -style call syntax seemed most people's favorite:

(call/kw proc 1 2 3 :d 4 :e 5 :f 6)

Again, :d vs d: vs #:d is up for debate (resolved below).

Other alternatives:

(lambda/kw (a b c : d e f) ...)

(call/kw foo 1 2 3 : c 4 d 5 e 6)
(call/kw foo 1 2 3 : c 4 : d 5 : e 6)

### R7RS-large and hygienic keywords

Marc Nieper-Wißkirchen expressed concern about what happens in
situations where keywords can be confused with identifiers from the
surrounding lexical environment. E.g.:

(let ((:e 3))
   (call/kw foo 1 2 :e 4))

Does the :e inside call/kw signify a keyword argument named `e`, or a
positional argument whose value comes from the lexical variable named `:e`?

Considerations:

* While a separate : identifier would work, no existing Lisp/Scheme
system with keyword arguments uses it. All of them use some variant of
:foo or #:foo with the colon joined to the identifier.

* Several Scheme implementations already read :e as a keyword object.
This means that in practice, when reading random Scheme code from the
internet, :e may already be intended as a keyword object. Hence using :e
as a variable name in a portable Scheme program may be confusing to
people or unreliable across existing Scheme implementations.

* However, no RnRS report has keyword objects yet so standard Scheme
reads :e as an ordinary identifier. It would be nice to preserve this
simplicity in the standard, and add an alternative syntax for keyword
objects. Kawa, Guile and Racket already have #:e.

* Allowing either ordinary identifiers or keyword objects as keyword
names permits hygienic keywords (which don't yet exist in any known
language, but could be added to Scheme implementations that want them).

I suggest we do the following:

* Since SRFI 177's main goal is compatibility with existing Scheme
keyword systems, it will use the syntax (call/kw foo 1 2 :d 3 :e 4).
Here, :d and :e can be either identifiers (in any Scheme implementation)
or keyword objects (in Scheme implementations that have them). In
identifiers, the colon is recognized as either a prefix or a suffix.
Since there are Scheme implementations that accept either or both, it's
clearest to allow all alternatives for users of the SRFI. call/kw would
scan for those literal symbols at read time, so the symbol :e inside the
lambda list would shadow any lexical variable named :e.

* SRFI 177 call/kw would translate all of e: and :e and #:e into the
non-hygienic, global keyword name `e`. So an `e` keyword used in one
Scheme library would be equal to the `e` keyword in any other library.
This is the case in Common Lisp, as well as in all Scheme
implementations that currently have keyword arguments.

* R7RS-large should probably not use SRFI 177. If it gets keyword
arguments, it could add the #:e standard syntax for global keyword
objects, and could use its own version of call/kw that accepts only
keyword objects. R7RS-large keyword lambdas should be specified so that:

- R7RS call/kw can call SRFI 177 keyword lambdas.

- SRFI 177 call/kw can call R7RS keyword lambas as long as all keywords
used are non-hygienic Common Lisp-style global keywords.

* While the R7RS-large keyword lambdas probably don't need to support
hygienic keywords, they should leave open the possibility that a
particular implementation may add them. This can be done by requiring
keyword objects and leaving the SRFI 177 symbol-parsing compatibility
hack out of the standard language.

### More complex keyword specifications

SRFI 177 is meant to be the simplest thing that could work. Hence its
lambda/kw purposefully leaves out features like:

* default values for keyword arguments
* mixing keyword arguments with positional optional arguments
* mixing keyword arguments with a rest argument
* allow-other-keys (collecting unknown keywords into a list)

However, SRFI 177 call/kw is fully capable of calling procedures defined
by other facilities (such as SRFI 89 or implementation-native lambda)
which use these features. Since SRFI 177 is compatible, its restricted
feature set does not present a problem in my opinion.

Once we decide the final form of SRFI 177, and how to tackle keywords in
R7RS-large, we have a good basis for writing a follow-up SRFI(s) that
allow more complex keyword lambdas to be defined. Perhaps SRFI 89 or
Gauche's syntax is already enough.

If Marc implements hygienic keywords, those would probably have to be
taken into account by specifying a version of call/kw that is different
from both SRFI 177 as well as R7RS-large. It would have to be different
because it needs a different way to detect which elements in the lambda
list are keywords. The (call/kw 1 2 3 : d 4 : e 5) syntax could handle
both hygienic and non-hygienic keywords, as could the original (call/kw
1 2 3 (d 4 e 5)). However, these syntaxes work differently from the
usual (call/kw 1 2 3 :d 4 :e 5) that Lisp/Scheme programmers have used
for decades. Since interest in hygienic keywords is limited so far, and
there are 10 Scheme implementations as well as Common Lisp, Emacs Lisp
and Clojure using the :e style keywords, I think we should go with that
convention for SRFI 177. It would probably be wise for R7RS-large as
well, but I'll leave that decision to others.