Hi!

Thanks for this SRFI. I do not like SRFI 64, and have been meaning to SRFI-fy my preferred alternative (chibi test) for a while, but I had got a bit hung up on its (imo) ugly internal use of ad hoc alists for test specification and result records. However, this looks potentially even better than (chibi test).

Some comments:

1. The strongly established convention for the names of variables which hold parameter objects in Scheme and Racket is that they should begin with current-. Therefore ‘test-runner*’ should be called ‘current-test-runner’.

I assume the asterisk is intended to indicate the parameter-ness of this variable, since there is no asterisk-free version. But asterisk has its own function in Scheme as a prime character on variants of existing forms, as in let and let*.

2. The convention (<<parameter object>> <<new value>>) to mutate a parameter object is not standard in R7RS small. Parameter objects’ values can only be set with the ‘parameterize’ syntax.

3. Like other commenters, I do not like the name ‘is’. My suggestion is to rename ‘is’ to ‘test’ and ‘test’ to ‘tests’. (Try it, with a useful description, and you’ll see what I mean. Hint: ‘tests’ is a 3.sg.act.ind verb, not a plural noun.)

4. The two forms of ‘is’ should probably have different names. I’m not sure how strongly I feel about this without trying to use it, but it feels unintuitive to me that an attempt to do something like (is (some-syntax some-subforms …)) will (I assume) fail because it will try to take the value of the identifier ‘some-syntax’ for its potential failure report.

The syntaxes should be, respectively, (is/apply <<procedure>> <<arg>> …) and (is/expr <<expression>>). Note the lack of parentheses around the procedure form, indicating that it itself must evaluate to a value in its own right, like the first argument to ‘apply’. The names, obviously, need bikeshedding.

5. I am concerned about the lack of explicit facility for some types of tests, specifically:

i. Testing that the right inexact number is returned. For this, a comparison predicate is needed which tests whether the returned value is within epsilon of the expected value.

ii. Testing that an exception of the right condition type is raised. I’m not sure how the current set of facilities can do this and report useful test failure results in the cases that no exception at all was raised, or an exception was raised but of the wrong kind. (Incidentally, this is yet another use case for a variant of with-exception-handler which allows to detect whether the exception is continuable, since the first exception raised may be continuable, but not the right kind yet …)

6. It would be nice if the SRFI or its sample implementation included test runners that generated nice terminal output (comparable to (chibi test)) plus either JUnit XML or some other machine-readable test report format. But this is very much on my ‘nice to have’ list, considering the sample implementation is only that – a sample.

Thanks again!

Daphne