On Sat, 6 Mar 2021 10:23:35 +0100
Amirouche Boubekki <xxxxxx@gmail.com> wrote:

> Le sam. 6 mars 2021 à 10:10, Marc Nieper-Wißkirchen
> <xxxxxx@nieper-wisskirchen.de> a écrit :
> >
> > It's not at all clear to me how the special forms you suggest fit into
> > the RnRS scheme model of reading, expanding, and executing a program
> > (with libraries).
> >
> > Can you explain the semantics in detail? Say for a file containing
> > "specials" like `#:` that is read through the `read` procedure. When is
> > `set-special!` evaluated and how and when do the implied reader
> > extensions apply?
>
> If a program.scm contains `set-special!` and use that reader extension
> defined in the very same file, how does it work or how can it be
> implemented?

If implemented properly by the Scheme system and not by the patching read it
should apply the syntax immediately after the code is executed. I'm not sure
how difficult it would be for other implementations to add new tokens in the
same file.

This how it works in my Scheme implementations, you can execute both lines in
same file:

(set-special! "#:" 'quote)
#:foo

But this is possible because of how I've implemented my parser and
interpreter, I use Lexer that read the tokens while it's parsing and eval
evaluates the S-Expression one by one using generators. So before next
S-Expression is parser the previous one is already evaluated.

But this will not be possible in my implementation:

(begin
  (set-special! "#:" 'quote)
  #:foo)

because this is single S-Expression that will be read by the Lexer then
parsed and evaluated as single expression.

I'm attaching the base implementation, where You need to call read that was
patched in order to see how this works.

We can simplify the SRFI by requiring for implementation to work in different
files. This is how it was working before I replaced tokenizer in my Scheme
that return list of all tokens for whole file with incremental Lexer that is
tokenizing while the code is parsed.

Specials was could not be in same file as the code that was using it, so it
was pretty limited.

The Lexer/Parser in my Scheme implementation works similar to eval/apply in
meta circular evaluator they both executing in a loop.

If you want to play with code you can install LIPS my scheme using NPM. If
you don't have Node.js you need to install it first and then call:

npm install -g @jcubic/xxxxxx@beta

with this you can create file test_specials.scm

(set-special! "#:" 'quote)
(display #:foo)
(newline)

and run `lips test_specials.scm`

When just patching read function it may not work like this if read just
tokenizing whole file and then parsing it. It would probably require lot of
code to make it work by patching the read function. But this can be left to
implementation to support this SRFI fully.

>
> > -- Marc
> >
> > Am Fr., 5. März 2021 um 21:44 Uhr schrieb Jakub T. Jankiewicz
> > <xxxxxx@onet.pl>:
> >>
> >> Hi,
> >>
> >> I plan to work on new SRFI I would know what you think about my idea.
> >> I've already implemented first idea in my Scheme based lisp called LIPS
> >> written in JavaScript.
> >>
> >> The feature I want to work on is called syntax extensions is the way to
> >> add new syntax like build in , ' ` ,@ that work exactly the same, that
> >> would allow users to modify the parser/reader. The feature work similar
> >> to compiler extensions in Common Lisp. You define the mapping - string
> >> of one or more characters and symbol for a function or macro that will
> >> be called when parser/reader will read the token with S-Expression that
> >> immediately follow the token.
> >>
> >> Example from my own code:
> >>
> >> (set-special! "#:" 'gensym-interal)
> >>
> >> (define (gensym-interal symbol)
> >>   "(gensym-interal symbol)
> >>
> >>    Parser extension that create new quoted named gensym."
> >>   `(quote ,(gensym symbol)))
> >>
> >> If gensym function is defined (most Scheme implementation support this or
> >> other similar function together with lisp macros).
> >>
> >> If you execute #:foo it will return unique gensym symbol.
> >>
> >> With this you can implement lot of features that are already in R7RS
> >> spec or in other SRFI documents. The syntax can be simplified to have
> >> single expression that define prefix and function. But I think this is
> >> better because you have normal function that can also be used elsewhere
> >> not only in parser.
> >>
> >> Another example is alias for quotation:
> >>
> >> (set-special! "’" 'quote)
> >>
> >> and this will allow to run every example in R7RS pdf that have different
> >> quote character that throw syntax error when copy pasting.
> >>
> >> Another use is to implement SRFI-10, vectors, byte vectors or
> >> All types of Typed vectors that I've found in this document:
> >>
> >> https://small.r7rs.org/wiki/NumericVectorsCowan/17/
> >>
> >> In my implementation I also have complementary feature that all to add
> >> string representation for objects. They are JavaScript objects so I just
> >> check the type.
> >>
> >> Example:
> >>
> >> (set-repr! Uint8Array (lambda (x)
> >>                         (string-append "#u8" (repr (u8vector->list x)))))
> >>
> >> This is code based on my standard library (in my code I have one big
> >> macro that define all different types from Numeric vectors by Cowan)
> >> that add support for printing byte vectors as "#u8(...)" which make the
> >> data type transparent to read and write.
> >>
> >> My implementation is kind of special because it's interpreter created in
> >> other runtime dynamic language (JavaScript). But this can be extended
> >> into records or different generic data types that can be easily identify
> >> by Scheme system.
> >>
> >> Why this is useful (if you don't think that it's useful already).
> >> It's because if this type of extensions are supported by the language (in
> >> form of SRFI) library can create missing syntax that are in other SRFI or
> >> future versions of Scheme. It can be useful to add new syntax. Of course
> >> the same as with macros or even more you should use those only when
> >> necessary, that would make the code simpler and easier to read.
> >>
> >> Those can be two different SRFI, the second (set-repr!) can be defined
> >> as a way to define representation for records that are new data type
> >> defined by R7RS, but can be left for implementers to add different
> >> objects that the Scheme system allow users to define, basic support can
> >> be for records.
> >>
> >> With records one can define their own list type (like example from R7RS
> >> document)
> >>
> >> (define-record-type <pare>
> >>   (kons x y)
> >>   pare?
> >>   (x kar set-kar!)
> >>   (y kdr))
> >>
> >> (set-special! "<>" 'k-list)
> >>
> >> (define (k-list rest)
> >>   (fold-left kons nil rest))
> >>
> >>
> >> (define (type->string x)
> >>   (if (pare? x)
> >>       (pare->string x)
> >>       (let ((p (open-output-string)))
> >>         (write x p)
> >>         (get-output-string p))))
> >>
> >> (define (pare->string pare)
> >>    (if (pare? pare)
> >>        (string-append "<>(" (pare->join pare) ")")
> >>        (type->string x)))
> >>
> >>
> >> (define (pare->join pare)
> >>    (if (pare? pare)
> >>       (string-append (type->string (kar pare))
> >>                      (if (pare? (kdr pare)) " " "")
> >>                      (pare->join (kdr pare)))
> >>       ""))
> >>
> >>
> >> Usage:
> >>
> >> (print (pare->string (kons 1 (kons 2 nil))))
> >> (print (pare->string <>(1 2 3 4)))
> >>
> >> and if set-repr! support records (in my implementation as I've said it
> >> only support JavaScript instances, but this is something I can easily
> >> add) the only thing that left is that you can execute this:
> >>
> >> (set-repr! <pare> pare->string)
> >>
> >> and you have fully working transparent new data type.
> >>
> >> Unfortunately you can't add support for <1 2 3> as representation and to
> >> make this align better with other SRFI we can suggest users to use #
> >> prefix for user data types, but I think that this should be only
> >> suggestion. User should be allowed to use any syntax he want.
> >>
> >> I want to know what you think about those proposals. I've already talk
> >> with Arthur A. Gleckler and he suggested that I should mention this here.
> >>
> >> --
> >> Jakub T. Jankiewicz, Web Developer
> >> https://jcubic.pl/me
> >>
>
>

--
Jakub T. Jankiewicz, Web Developer
https://jcubic.pl/me