On 2026-04-04 12:00, Daphne Preston-Kendal wrote:

> Hi!
>
> Thanks for this SRFI. I do not like SRFI 64, and have been meaning to
> SRFI-fy my preferred alternative (chibi test) for a while, but I had
> got a bit hung up on its (imo) ugly internal use of ad hoc alists for
> test specification and result records. However, this looks potentially
> even better than (chibi test).

Hello Daphne!

Thank you very much for kind words! :)

>
> Some comments:
>
> 1. The strongly established convention for the names of variables
> which hold parameter objects in Scheme and Racket is that they should
> begin with current-. Therefore ‘test-runner*’ should be called
> ‘current-test-runner’.
>
> I assume the asterisk is intended to indicate the parameter-ness of
> this variable, since there is no asterisk-free version. But asterisk
> has its own function in Scheme as a prime character on variants of
> existing forms, as in let and let*.

Yeah. I found a lot of current- variables in Guile not being paramaters,
and I decided to go with more explicit naming like
*dynamically-scoped-var*, but later I realised that it's hard to
prefix-autocomplete if there is a leading '*', so I left only trailing
asterisk.

I don't have a strong preference here, current-test-runner will work
well for me too.

BTW, I thought that more correct name will be something like
test-manager or test-superviser as it not only runs tests, but also
loads and schedules. And loading in fact is completely distinct phase,
not necessary followed by running.

WDYT?

>
> 2. The convention (<<parameter object>> <<new value>>) to mutate a
> parameter object is not standard in R7RS small. Parameter objects’
> values can only be set with the ‘parameterize’ syntax.

Actually, it doesn't have to be a parameter, it can be fluid or even
something else.  There are two primary use cases I have in mind:
- Swapping a global default test runner, which will be
loading/collecting defined tests and later maybe running them.
- Temporary overriding test runner to be able to test the testing
library itself or do some other kind of advanced tech.

Not being able to set test-runner parameter permanently can be kinda
restricting and break a few workflows I had in mind.  But I'll think
about it and report later on that.

>
> 3. Like other commenters, I do not like the name ‘is’. My suggestion
> is to rename ‘is’ to ‘test’ and ‘test’ to ‘tests’. (Try it, with a
> useful description, and you’ll see what I mean. Hint: ‘tests’ is a
> 3.sg.act.ind verb, not a plural noun.)

suite->test->assert hierarchy mimics JUnit and also some other
popular frameworks.

While check or test can be a better name semantically then is,
suite->tests->test will be hella confusing.

Also, keep in mind that there is a loading phase and at first we lift
suites and tests into memory. And during the second phase we schedule
and run tests.

According to is naming. Try to play with it.  Read some test in suitbl
library.  And report how you feel about it.  Also, take a look a
rationale I wrote in the other thread in this mailing list. And let me
know what you think.

>
> 4. The two forms of ‘is’ should probably have different names. I’m not
> sure how strongly I feel about this without trying to use it, but it
> feels unintuitive to me that an attempt to do something like (is
> (some-syntax some-subforms …)) will (I assume) fail because it will
> try to take the value of the identifier ‘some-syntax’ for its
> potential failure report.

Yeah this is a known and documented limitation:
https://git.sr.ht/~abcdw/guile-ares-rs/tree/nil/tests/guile/ares/suitbl/definitions-test.scm#L131

Personally, I can live with it, you can rewrite the assert the way it
won't use macro inside. And I think I even mentioned it in a spec.

>
> The syntaxes should be, respectively, (is/apply <<procedure>> <<arg>>
> …) and (is/expr <<expression>>). Note the lack of parentheses around
> the procedure form, indicating that it itself must evaluate to a value
> in its own right, like the first argument to ‘apply’. The names,
> obviously, need bikeshedding.

This is an actually cool idea and it also removes a level of nestness.

Another option is just to disallow the (is expr) usage at all (it's
rarely useful anyway) and you can always write (is (true? expr)) as an
escape hatch.

>
> 5. I am concerned about the lack of explicit facility for some types
> of tests, specifically:
>
> i. Testing that the right inexact number is returned. For this, a
> comparison predicate is needed which tests whether the returned value
> is within epsilon of the expected value.

;; 0.01 is epsilon, can be an optional argument
(is (inexact-equal? 0.5 return-value 0.01))

Like this?

>
> ii. Testing that an exception of the right condition type is
> raised. I’m not sure how the current set of facilities can do this and
> report useful test failure results in the cases that no exception at
> all was raised, or an exception was raised but of the wrong
> kind. (Incidentally, this is yet another use case for a variant of
> with-exception-handler which allows to detect whether the exception is
> continuable, since the first exception raised may be continuable, but
> not the right kind yet …)

(test "expression throws programming-error on unbound variable"
  (is (throws-exception? (+ b 1 2) programming-error?)))

https://git.sr.ht/~abcdw/guile-ares-rs/tree/10850b080bbfe901ca7c4b557bdeaf08c1da78d7/tests/guile/ares/suitbl-test.scm#L286

Possible implementation:
https://git.sr.ht/~abcdw/guile-ares-rs/tree/10850b080bbfe901ca7c4b557bdeaf08c1da78d7/src/guile/ares/suitbl.scm#L87

A helper for checking an exception message:
https://git.sr.ht/~abcdw/guile-ares-rs/tree/10850b080bbfe901ca7c4b557bdeaf08c1da78d7/tests/guile/ares/suitbl-test.scm#L150

Example with a hand-crafted specific "type" of exception:
https://git.sr.ht/~abcdw/guile-ares-rs/tree/10850b080bbfe901ca7c4b557bdeaf08c1da78d7/tests/guile/ares/suitbl-test.scm#L297

>
> 6. It would be nice if the SRFI or its sample implementation included
> test runners that generated nice terminal output (comparable to (chibi
> test)) plus either JUnit XML or some other machine-readable test
> report format. But this is very much on my ‘nice to have’ list,
> considering the sample implementation is only that – a sample.

We already have nice reporters in suitbl library: verbose, minmal,
junit.  Check out the videos mentioned at the end of SRFI, there are
some cool demos and use cases, the reporters are also visible.

https://youtu.be/JI7Z6-tndaw

I didn't reply for a couple days, because I wanted to finish refactoring
of the current code base and implement one more cool reporter.  But it
still requires a few more days :D

Please, share your thoughts after a second round of review, will be
really glad to hear! :)

--
Best regards,
Andrew Tropin