>     Magic comments are exactly what I wanted to avoid.
>
> They are not bad per se.
>
>     - The Lisp family is well known for a syntax that is uniquely suited to
>     be arbitrarily extended in a clean way.
>
> No; the lexical syntax is not and cannot be arbitrarily extended in a
> clean way. It's the *structure*, into which the syntax is parsed, which
> is quite uniform in Lisps.

> Rightfully so because the actual top-level program that will be run
> should not be bothered with artifacts from the script launcher.

> You can't `(read)' the data because the lexical syntax of the script
> launcher s-expression may not coincide with the lexical syntax of the
> s-expression of a particular Lisp. You can't even tell a Scheme's `read'
> to parse just the first 4096 characters. etc-

> If they are really hard to parse, it will have nothing to do with that
> they have to appear in lines that have to prepended with a ";" in the
> first column...

> It breaks down once a future dialect wants to change the s-expression
> syntax (e.g. the lexical syntax of identifiers) in some way not yet
> predictable.

We approach the problem from quite different perspectives, which is why
we find it hard to communicate.

My overriding concern, as usual, is long-term cultural optimization. The
entire "ecosystem" of Scheme and Lisp standards, implementations, and
tools should be something that is easy to comprehend, and should be
simple within reason. If this needs to be done at the expense of some
generality or near-term convenience, I'm happy to pay that price.

If script writers agree to use (declare-file ...) in a way that suits
the Scheme/Lisp standards/implementations being targeted, and the syntax
understood by the launcher, as well as editors like Emacs, this means
all of these tools can make use of the same script file as-is. The tools
are simpler and the way they work is easier to comprehend since they do
fewer implicit operations.

Magic comments show that people have failed to come together to solve a
cultural problem via planning and cooperation, and have to resort to a
technical hack which is sub-optimal for people and tools alike. Similar
comments apply to stuffing code in strings (GCC inline assembly), etc.

> So you are inventing your own s-expression syntax. That's fine and by
> all means, use it for the script format. But don't restrict all
> s-expressions in the script files to be of this format. A script file
> need not and should not be parsable with any `read' if you want to
> support more than one Scheme dialect.

Defining a new subset, not inventing new extensions. Big difference. A
subset doesn't create a new artifact, just restricts some uses of an
existing artifact to gain more value than is lost by the restriction. I
love subsets as a problem-solving tool.

The launcher should stop reading as soon as it has found (declare-file
...) so that the rest of the file is free to use richer lexical syntax.
I agree that restricting the entire script to least-common-denominator
S-expressions is far too prohibitive.

>     What do you think about launching a Scheme program from a tar file,
>
> One can certainly add this. But this is not a complete substitute
> because humans can read text files but not tar files.

A Scheme launcher with built-in noweb-style text substitution still
seems like a weird factoring of the problem to me. Since noweb is
already established, would a generic "noweb launcher" supporting any
language (including Scheme) be reasonable? Using noweb would enable the
script to double as a literate program, too. I bet literate programming
fans would love writing their scripts in this style.

Since our present conversation can't come to an agreement, I suggest
that I polish lila into a release-quality tool, using its current
approach of parsing a lenient S-expression subset at the start of the
file. Since lila is not meant to be a standard (in the sense of SRFI or
scheme-script), people can ignore it if they don't like it, and it
doesn't encroach on standard Scheme territory by using a command name
with `scheme` in it.

If you like, you could then show cases that lila solves badly (i.e. the
restrictions of the S-expression subset are too severe, and prevent
solving practical problems). Or think of an alternative approach and/or
implement something.

It seems we have trouble involving more people in the discussion without
complete, implemented proposals to show.