Re: Scheme portable testing prior art
Lassi Kortela 20 Aug 2020 19:17 UTC
> There is also SRFI 78, which, I think, Wolfgang mainly uses.
Yes, 78 exists but 64 is far more widely used. 78 is much simpler.
> I don't understand this problem. If you maintain a Scheme
> implementation and want to ship SRFI 64 but don't like the default
> test runner, you can always ship with an adapted default test runner.
SRFI 64 gives a somewhat extensive test runner API. It may be that
people are opposed to that API; I don't know. Schemes tend to have their
own runners from long ago. It's true that the UI can be made different.
Whatever the reasons, a full-featured test definition framework is much
simpler than a full-featured test runner. There's also much less variety
in definition frameworks. Therefore, to maximize adoption among Scheme
implementations, it would seem to split off the definition framework and
let people plug it into whichever runner they want.
> I am not sure whether writing just another test SRFI will improve the
> situation, at least as long as it is not backward compatible with an
> existing one.
>
> So if we write a new SRFI, I think it should be backward compatible
> with SRFI 64 so that every SRFI 64 test-suite will still be a SRFI 264
> test-suite.
Fully agreed. That was exactly my point.
We can debate about whether test-begin/test-end should be supported,
since many people seem to dislike them, preferring test-group. But
that's a detail.
> PS Guile and autoconf now have great support for SRFI 64. The test
> results are reported through a TAP interface.
>
> I have also written a SRFI 64 implementation (based on Per's), which
> directly outputs its results in TAP, which can be grokked by other
> build tools. So we have at least three SRFI 64 implementations, which
> is a good thing.
Sounds very good. Thanks for your work!