On 2026-06-24 11:56, Wolfgang Corcoran-Mathe wrote:

> On 2026-06-23 19:07 +0700, Andrew Tropin wrote:
>> On 2026-06-20 13:55, Wolfgang Corcoran-Mathe wrote:
>> > 6. There seems to be no way to *run* a test or suite!  I searched in
>> > vain for runner/run-test and runner/run-suite messages.  The SRFI is
>> > quite clear that runner/load-test and runner/load-suite are "loading
>> > form[s]" that do not execute anything.
>>
>> Good catch. You can take a look at suitbl library, it implements
>> `auto-run?` configuration option, which runs tests immediately after
>> they loaded, but also allows to send a separate message.
>>
>> Probably, it's better to define a message type or provide some
>> recommendations on this regard in SRFI.  I wrote a not for myself for
>> the next revision.
>
> I'm trying to add such messages (runner/run-test & runner/run-suite,
> I've called them) to my sample implementation, but I noticed that we
> have no reliable way to name a loaded test.  A constructor for a
> basic runner which allows loading & running tests (& only tests, for
> simplicity) might look like this:
>

A few notes.

I suggest:
1. runner/run-test accepts the whole test entity alist.

1.1 Originally, I did a separate message handler, but later after a few
long refactoring sessions we switched to a simple function and kept only
runner/run-tests

https://git.sr.ht/~abcdw/guile-ares-rs/tree/d549104b/src/guile/ares/suitbl/runner.scm#L152

2. ommit runner/run-suite, as it just an edge case of runner/run-tests.

3. runner/run-tests accept a runner config, which includes
schedule-tests options:

https://git.sr.ht/~abcdw/guile-ares-rs/tree/d549104b/dev/guile/demo/2026-03-30-suitbl.scm#L320

This way you can select which tests and in which order to execute.
You can select by regexp, sort by previous execution time, etc, etc.

>     (define (make-simple-test-runner)
>       (let ((loaded-tests '()))
>         (lambda (msg)
>           (let ((type (message-type msg)))
>             (case type
>               ((runner/load-test)
>                (let ((t (message-ref msg 'test)))
>                  (set! loaded-tests (cons t loaded-tests))))
>               ((runner/run-test)  ; not in the SRFI
>                ;; ???  What is a test ID? ???
>                (let ((id (message-ref msg 'test-id)))
>                  (run-test (get-test-by-id loaded-tests id))))
>               ;; ...
>               (else (error 'simple-test-runner
>                            "unhandled message type"
>                            type
>                            msg)))))))
>
> Clearly there's no notion of a "test ID" in the current spec, and thus
> no way for the run-test message to name the test it wants run.  The
> only thing we can do is run *all* loaded tests.  That's too clumsy to
> warrant the load / run decoupling, in my opinion.

We have ability to run specific tests, see schedule-tests usage examples
in suitbl library.

>
> In short, it seems that each test or suite needs a unique name.  This
> could be provided by the test writer, although there will be no
> guarantee of uniqueness.

I agree that ID would be benifitial.  I have use cases, where it's
needed and we have an internal enumeration to make test with the same
description distingisheable:

https://git.sr.ht/~abcdw/guile-ares-rs/tree/nil/src/guile/ares/suitbl/ares.scm#L66

It's required for nREPL frontend/IDE, where we have only serialized
representation of the test, which we can't compare for equality.
Probably there are other use cases, where it can be useful.

> The runner could also generate an ID on receiving a runner/load-test
> (or runner/load-suite) message, and return it to the sender of the
> message.  The runner, with knowledge of its loaded-test database could
> guarantee a unique ID.  But the idea of 'test' and 'suite' forms
> returning values cuts against the SRFI's design.

1. Yes, we can add test/id based on internal test-runner state.
2. We don't need to communicate it back to definition code.  The only
responsibility of definition code is to lift suite/test and send it to
test-runner for load.

It's not necessary to define it as a part of the SRFI-269, as it's
completely a runner's side of things, but probably worth mentioning.

"Note for test runner implementers: it's recommended to associate a
unique test/id and suite/id for any loaded entity. To make it uniquely
identifieable and serializeable, blablabla."

Something like that I guess.

I added a note to my SRFI-269 backlog.

If you have any particular ideas on how to better implement IDs, please
let me know.

--
Best regards,
Andrew Tropin