Text substitution macros and multi-file archives Lassi Kortela 28 Jun 2021 07:24 UTC

> Here, the implementation for a hypothetical "foo-scheme" wants to make
> use of special lexical syntax, namely `##foo', of which
> `scheme-script-ce' may have no idea so that the above cannot work. A
> similar problem occurs when we want to mix R6RS and R7RS versions of a
> program; in the R6RS version, `#vu8(' is used to introduce bytevectors
> vs. `#u8' for the R7RS world.
>
> So what we do need is a capability of raw textual substitution in
> ce-scripts, e.g. as follows:
>
> #!/usr/bin/env scheme-script-ce
> (cond-expand
>    ((foo-scheme)
>     (include-chunk <<foo>>))
>    (else
>      ...))
> #!eof
> <<foo>>=
> ##foo
> @
>
> Here, I have used special lexical syntax (to be understood by
> `scheme-script-ce') to mark the end of the list of external
> representations of Scheme datums. Afterward, chunks in the style of the
> "noweb" program follow. (I have used its syntax just for demonstration
> purposes so that I didn't have to invent my own.)
>
> `include-chunk' will insert the referenced text into the final program
> that is fed by `scheme-script-ce' to a supported implementation.

Free-text substitution macros tend to confuse humans and text editors
alike. It would be a good principle for each Scheme implementation to
have an alternative way to use the features that are invoked via
non-standard read syntax.

For example, instead of #/.../ for a regexp literal, a portable script
should use (regexp-compile ...), even when `regexp-compile` is an
implementation-specific procedure called inside `cond-expand`, so that
the other implementations that can run the script can still read it.

Bundling multiple Scheme files into something like a `shar` archive
would surely be useful sometimes, but could we solve that problem by
permitting multiple (define-library ...) forms as you suggest, instead
of concatenating free-form files? Maybe those (define-library ...) forms
can embed non-Scheme files as string or bytevector literals if they need
them.

The JavaScript community has a lot of experience with transpilers like
Rollup (https://www.rollupjs.org/) that bundle modules into one script.
Writing a similar one for Scheme would be an interesting exercise for
which Unsyntax could be a good foundation. It should also be possible
possible to compile most R7RS code into a form that doesn't use
(define-library ...) and relies on (let ...) scoping only, renaming
identifiers as needed to avoid clashes.