Re: Repository of R7RS implementations and tests Ciprian Dorin Craciun 28 Mar 2018 19:41 UTC
On Wed, Mar 28, 2018 at 9:13 PM, <firstname.lastname@example.org> wrote: >> Yes, I know about that, and its the first link on my list. > > Read again, https://ecraven.github.io/r7rs-coverage > > coverage, not benchmarks ;) Sorry I miss-read. :) Indeed it ticks all my "requirements", as in they are simple to implement support for them. Are you open for a collaboration in filling them in? And also creating a "small" standard to describe such tests? I have extensive tests for some functions you are currently missing, like you can see bellow: https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/control.sst https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/ports-memory.sst https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/ports-native.sst https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/quotation.sst https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme/strings.sst Other tests are found at: https://github.com/cipriancraciun/vonuvoli-scheme/blob/master/tests/scheme (Symbols starting with `~` are non R7RS which can be removed; and there are some extensions particular for my implementation like `#void` and `#string( ... )` which can be easily converted.) I like the fact that you can "group" the tests, and the report you generate based on them, but it seems it relies on specially crafted comments. If you are open I would propose two minor additions (and a few clarifications): * base the grouping on s-expressions, something like `(test-group NAME test-statement ...)`; (where NAME can be any valid S-expression, and if it is a symbol it should be quoted; this is so that one can implement it with a plain lambda;) * introduce a few testing primitives for three (or four) common cases: * `(test= <expression> <expected> [<equal?>])` -- which would allow the test harness to print something like: `test returned x, expected y`; * `(test <expression> [<predicate?>])` -- which would be an extension equivalent to `(test (<predicate?> <expression>))` but would allow the harness to print something like `expression returned x which doesn't satisfy <predicate?>); * `(test-error <expression> [<predicate?>])`; * the `test` constructs have to be found at the top level; (so that one can parse and transform them into test cases native to the system at hand without knowing anything about the tested language;) * other statements are allowed at top level; * everything is executed in top-down manner; Implementing these constructs is trivial, and doesn't require anything else except lambdas. Moreover the clarifications allow the test files to be treated just like simple Scheme source files, meanwhile also allowing native test harnesses to create specific test cases. I use the prefix `test` instead of `assert`, because most likely many Scheme implementations would have `assert` as actual run-time assertions, and using them would mean that the test harness would have to "override" their meaning. Moreover even inside tests, one could use `assert` to check for invariants. (Think of a loop that computes a value: the `test` construct should check that the final outcome is correct, meanwhile the `assert` construct could be used to double check some internal invariants at each cycle.) I know about SRFI 64 and 78, and in fact my three constructs seem similar to the simple ones proposed in SRFI 64, although my proposed variant is compatible to what we both already have in our tests (and it feels much more natural to say `(test <expression> <expected>)` than the other way around `(test <expected> <expression>)`). Ciprian.