RE: Repository of R7RS implementations and tests Ciprian Dorin Craciun 28 Mar 2018 14:47 UTC

[Please don't hate me for asking so many questions.]  :) :)

So I was going through the `srfi-discuss` mailing list archive and
stumbled upon an interesting thread and project:

  https://srfi-email.schemers.org/srfi-discuss/msg/2869481
  https://github.com/TaylanUB/scheme-srfis

So I wondered what happened in the interim with this project?

[In the next section I'm mainly referring to the subject of R7RS and
SRFI compliance testing, especially for "young" Scheme
implementations.]

I am especially interested in the test cases for SRFI's and especially
for R7RS (which includes some SRFI's), because I am writing an R7RS
scheme implementation and I want to check that it is compliant with
the standard.  However I have yet to find an easily integrateable test
suite.

For R7RS I tried to do my homework and searched for other
implementations out there and looked at their test suites.  For the
reference this is what I found:

    https://github.com/ecraven/r7rs-benchmarks
    https://github.com/larcenists/larceny/tree/master/test/R7RS
    https://github.com/ashinn/chibi-scheme/blob/master/tests
    https://github.com/fadein/chicken-core/blob/master/tests
    https://github.com/justinethier/cyclone/tree/master/tests
    https://github.com/justinethier/husk-scheme/tree/master/tests
    https://github.com/leftmike/foment/tree/master/test
    https://github.com/shirok/Gauche/tree/master/test
    https://github.com/lexi-lambda/racket-r7rs/tree/master/r7rs-test/tests/r7rs/chibi
    https://github.com/picrin-scheme/picrin/tree/master/t
    https://github.com/aoh/owl-lisp/tree/master/tests
    https://gitlab.com/kashell/Kawa/tree/master/testsuite
    https://bitbucket.org/ktakashi/sagittarius-scheme/src/default/test/r7rs-tests
    https://git.savannah.gnu.org/cgit/guile.git/tree/test-suite
    https://code.call-cc.org/cgi-bin/gitweb.cgi?p=chicken-core.git;a=tree;f=tests

However in the end I decided to write my own test-cases, mainly
because running the tests meant having a fully working R7RS
implementation (especially including syntaxes), which is not the case
(yet)...

Given that any new Scheme implementation starts with simple
constructs, from simple syntaxes (`if`, `cond`, `define`) and simple
procedures (`+`, `cons`, `list`, etc.), and only later (if ever)
introduces hygienic macros or continuations, I would have expected to
find a set of tests that one could gradually integrate in its
implementation.

Therefore unfortunately I decided to create a custom, but simple
syntax, like `expression => expected` (but I could have settled for
something like `(test expression expected)`), for my tests.  The
advantage being that it required 5-minutes of investment in creating
the initial test harness, and I could have started tests from day one,
when the implementation contained only the parser and trivial
evaluator.  For example:

  https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/arrays.sst

So, after this long detour in which I tried to explain the context, I
would like to ask the Scheme community if there exist such
"standardized" test suite, especially for R7RS but also for SRFI's.

And if not is the community interested in creating one?

I can provide the test cases pertaining for R7RS that I have created
so far, and I could try to provide the "harness" to run it (against at
least Chibi and Gauche which I managed to compile and run on my
Linux).  However I can't provide all the test-cases myself.

Ciprian.

P.S.:  Is there a generic Scheme related mailing list where such
discussions can be held?  I think especially this email is out of the
scope of the SRFI workgroup...