Shrinking and nonfeatures
Antero Mejr 22 Mar 2024 00:15 UTC
I sent a pull request for draft v5 with a few small changes to the
wording and test suite. The main change is that the SRFI now explicitly
allows for, but does not require, shrinking. The sample implementation
does not use it.
I think it would be good to enumerate here what is purposefully not
included in the SRFI and why.
1. There are a multiple ways of doing shrinking for each generator type,
and the "best" way can be arbitrary or dependent on the code under test.
So it would be beyond the scope of a SRFI. For implementations that want
to include it, I would look at MacIver and Donaldson's paper on internal
test case reduction.
2. Targeted property testing is not included, as it is not commonly
used. It is not worth the additional interface complexity and
implementation burden, in my opinion.
3. Caching test inputs and saving/loading them from a database is not
included. It would create issues with portability of database formats,
performance, and handling of state. It is also not a common requirement.
All these features can be provided by an extension library without
changes to the base SRFI 252 interface. And if there are any concerns
with draft v5, please let me know here.
Thanks,
Antero