Re: Gathering comprehensive SRFI test suites in one place
Per Bothner 26 Jan 2020 19:32 UTC
On 1/26/20 11:10 AM, Peter Bex wrote:
> A quick test seems to indicate that at least the srfi-64 CHICKEN egg
> doesn't give any output when the tests pass. That's a bit confusing and
> annoying. If a test exits early, you can't really tell.
That is not how the reference implementation should work.
Typical output on Kawa:
$ ../bin/kawa "./lib-test.scm"
%%%% Starting test libs (Writing full log to "libs.log")
# of expected passes 269
# of expected failures 9
This is using the default test-runner, which writes a summary to standard output
(including 1 line for each unexpected failure or pass), and a detailed log to a file:
Kawa also has another test framework, which is useful for
short simple tests: The file is just an executable module,
with expected output in comments:
(let ((x ::int 1))
(set! x 0)
(try-finally
(if x (display "x is true")
(display "x is not true"))
(format #t "~%finally-clause executed~%")))
;; Output: x is true
;; Output: finally-clause executed
Comments can use plain text or regexps, specify error messages,
and compiler options. This format is convenient for simple
regression tests, but matching error messages is likely
to be implementation-specific.
(Kawa also has another pre-srfi-64 test framework, but that
is not used for new tests.)
For a portable testing framework, I recommend srfi 64.
The reference implementation is a bit complicated because
it has a lot of bells and whistles, partly for the sake
of the detailed log written to the log file.
However, a minimal implementation that does not support
custom test runners would be quite simple.
--
--Per Bothner
xxxxxx@bothner.com http://per.bothner.com/