A plea to make tests a required part of sample SRFI implementations noosphere@xxxxxx (17 Nov 2020 15:56 UTC)
Re: A plea to make tests a required part of sample SRFI implementations Wolfgang Corcoran-Mathe (17 Nov 2020 23:32 UTC)

A plea to make tests a required part of sample SRFI implementations noosphere@xxxxxx 17 Nov 2020 15:56 UTC

I've been porting a lot of SRFI's to Chicken lately, and noticed that
while many of the SRFI's I've ported have tests as part of their sample
implementation, some do not, while others don't have complete tests.

According to the srfi-process page[1]:

"The sample implementation should include automated tests. Tests make
porting to new Scheme implementations easier. They also help users
understand how your SRFI is to be used. However, if the sample
implementation is trivial or not really meant to be used, i.e. it is
just a proof of concept, it's okay to omit tests. That should be a rare
case. No specific test framework is required, but both SRFI 64 and SRFI
78 are available."

But clearly this encouragement to include automated tests is not enough
for some SRFI authors, who release their SRFIs either without automated
tests or with incomplete tests.

In my experience, insufficient testing has sometimes allowed bugs
(either in the SRFI reference implementation itself or in the port)
to slip through.

While I would be the last one to wish more work on anyone or put up more
barriers to making a SRFI available, but the fact is that SRFI authors
are the ones who are most intimately familiar with their specifications
and are in the best position to write tests that are robust and truly
exercise all the subtle corner cases that users or implementors of their
SRFI might run in to.

Having automated tests would also help SRFI authors work out bugs
in their own implementations before they are finalized, rather than
waiting until someone stumbles upon them in the wild.

I would like to ask the rest of the Scheme community what they think of
making automated testing a required part of SRFI sample implementations.

Sincerely,

Sergey Goldgaber

[1] - https://srfi.schemers.org/srfi-process.html