Wrapping up SRFI 177: Portable keyword arguments Lassi Kortela 22 Feb 2020 00:48 UTC
I'm ready to wrap up this SRFI as soon as we decide on what to do. ### Naming There was wide agreement: * Rename keyword-lambda to lambda/kw. * Rename keyword-call to call/kw. I'll also add define/kw, which is the obvious combination of define and lambda/kw. ### Performance, aka should define/kw make lambdas or macros? I designed lambda/kw as making real lambdas (using a rest argument for the keywords in Scheme implementations that don't have real keyword args) for simplicity: if keyword lambdas behave like ordinary lambdas in as many ways as possible, there is the least potential for surprises, and maximally easy to add keyword arguments to existing Scheme procedures that don't have any yet. Marc Nieper-Wißkirchen suggested that define/kw could make macros instead of lambdas, which would make keyword calls easier to optimize. That strategy makes me a bit uneasy since making keyword lambdas subtly different from other lambdas makes it a more complicated abstraction, with potentially more edge cases that cause trouble to some users. However, I don't have any particular concrete problems in mind; simplicity is just a design instinct. A good point of comparison is when OpenBSD designed the strlcpy() and strlcat() safer string functions for libc. They could certainly have better characteristics, but the priority was to design drop-in replacements for strcpy() and strcat(). I had a similar design mentality of trying to make lambda/kw a drop-in replacement for lambda. Common Lisp compilers can optimize keyword calls, but custom compiler macros may have to be written. Gambit, Kawa and Racket probably have some optimizations of their own. I'm not really qualified to judge the issue. Preliminary benchmarks (making lambdas with way too many keyword arguments) showed predictable slowdown in the portable implementation, but calls were still reasonably fast. They might not be fast enough in a tight inner loop; my recommendation would be to avoid fancy coding techiques such as keyword arguments in the hot path. FWIW, Dybvig is generally cautious about adding new language features to Chez Scheme since their performance implications are unclear. ### Syntax The current syntax is (lambda/kw (a b c (d e f)) ...) and (call/kw 1 2 3 (d 4 e 5 f 6)). It's a non-ideal compromise to allow implementations using only syntax-rules. We came to the conclusion that almost all Schemes support another macro system, so we agreed to drop the syntax-rules compatibility requirement. That gives more design freedom. Following that decision, this was the favorite lambda syntax: (lambda/kw (a b c :key d e f) ...) The precise naming of :key vs key: vs &key is up for debate. This Common Lisp -style call syntax seemed most people's favorite: (call/kw proc 1 2 3 :d 4 :e 5 :f 6) Again, :d vs d: vs #:d is up for debate (resolved below). Other alternatives: (lambda/kw (a b c : d e f) ...) (call/kw foo 1 2 3 : c 4 d 5 e 6) (call/kw foo 1 2 3 : c 4 : d 5 : e 6) ### R7RS-large and hygienic keywords Marc Nieper-Wißkirchen expressed concern about what happens in situations where keywords can be confused with identifiers from the surrounding lexical environment. E.g.: (let ((:e 3)) (call/kw foo 1 2 :e 4)) Does the :e inside call/kw signify a keyword argument named `e`, or a positional argument whose value comes from the lexical variable named `:e`? Considerations: * While a separate : identifier would work, no existing Lisp/Scheme system with keyword arguments uses it. All of them use some variant of :foo or #:foo with the colon joined to the identifier. * Several Scheme implementations already read :e as a keyword object. This means that in practice, when reading random Scheme code from the internet, :e may already be intended as a keyword object. Hence using :e as a variable name in a portable Scheme program may be confusing to people or unreliable across existing Scheme implementations. * However, no RnRS report has keyword objects yet so standard Scheme reads :e as an ordinary identifier. It would be nice to preserve this simplicity in the standard, and add an alternative syntax for keyword objects. Kawa, Guile and Racket already have #:e. * Allowing either ordinary identifiers or keyword objects as keyword names permits hygienic keywords (which don't yet exist in any known language, but could be added to Scheme implementations that want them). I suggest we do the following: * Since SRFI 177's main goal is compatibility with existing Scheme keyword systems, it will use the syntax (call/kw foo 1 2 :d 3 :e 4). Here, :d and :e can be either identifiers (in any Scheme implementation) or keyword objects (in Scheme implementations that have them). In identifiers, the colon is recognized as either a prefix or a suffix. Since there are Scheme implementations that accept either or both, it's clearest to allow all alternatives for users of the SRFI. call/kw would scan for those literal symbols at read time, so the symbol :e inside the lambda list would shadow any lexical variable named :e. * SRFI 177 call/kw would translate all of e: and :e and #:e into the non-hygienic, global keyword name `e`. So an `e` keyword used in one Scheme library would be equal to the `e` keyword in any other library. This is the case in Common Lisp, as well as in all Scheme implementations that currently have keyword arguments. * R7RS-large should probably not use SRFI 177. If it gets keyword arguments, it could add the #:e standard syntax for global keyword objects, and could use its own version of call/kw that accepts only keyword objects. R7RS-large keyword lambdas should be specified so that: - R7RS call/kw can call SRFI 177 keyword lambdas. - SRFI 177 call/kw can call R7RS keyword lambas as long as all keywords used are non-hygienic Common Lisp-style global keywords. * While the R7RS-large keyword lambdas probably don't need to support hygienic keywords, they should leave open the possibility that a particular implementation may add them. This can be done by requiring keyword objects and leaving the SRFI 177 symbol-parsing compatibility hack out of the standard language. ### More complex keyword specifications SRFI 177 is meant to be the simplest thing that could work. Hence its lambda/kw purposefully leaves out features like: * default values for keyword arguments * mixing keyword arguments with positional optional arguments * mixing keyword arguments with a rest argument * allow-other-keys (collecting unknown keywords into a list) However, SRFI 177 call/kw is fully capable of calling procedures defined by other facilities (such as SRFI 89 or implementation-native lambda) which use these features. Since SRFI 177 is compatible, its restricted feature set does not present a problem in my opinion. Once we decide the final form of SRFI 177, and how to tackle keywords in R7RS-large, we have a good basis for writing a follow-up SRFI(s) that allow more complex keyword lambdas to be defined. Perhaps SRFI 89 or Gauche's syntax is already enough. If Marc implements hygienic keywords, those would probably have to be taken into account by specifying a version of call/kw that is different from both SRFI 177 as well as R7RS-large. It would have to be different because it needs a different way to detect which elements in the lambda list are keywords. The (call/kw 1 2 3 : d 4 : e 5) syntax could handle both hygienic and non-hygienic keywords, as could the original (call/kw 1 2 3 (d 4 e 5)). However, these syntaxes work differently from the usual (call/kw 1 2 3 :d 4 :e 5) that Lisp/Scheme programmers have used for decades. Since interest in hygienic keywords is limited so far, and there are 10 Scheme implementations as well as Common Lisp, Emacs Lisp and Clojure using the :e style keywords, I think we should go with that convention for SRFI 177. It would probably be wise for R7RS-large as well, but I'll leave that decision to others.