Comments on Arthur's test framework
Lassi Kortela 22 Aug 2020 08:55 UTC
> * I use my own test framework, attached, which was inspired by the
> original version of JUnit. Its important features are:
> * Every test is lexically enclosed.
What does this mean precisely?
> * Every test or group of tests has a name.
Testcase names are optional in SRFI 64 and C/C. IIRC group names are
required.
> * Tests are first-class objects.
This is nice. SRFI 64 uses "test specifiers" to refer to tests, I can't
find objects in the public interface.
Test objects should probably go into a distinct SRFI from the runners
and the definition framework. It could be a test middleware in a similar
vein as WSGI/Rack/Ring are HTTP middleware.
> * One can run tests in a mode where only failures are reported. This
> way, one doesn't have to wade through output in order to figure out
> whether everything passed, or what failed.
Also nice, for the runner.
> * It's possible to run individual tests or test groups or all defined
> tests.
> * Test groups can be defined concisely.
In SRFI 64 and C/C this is `test-group` or `test-begin/test-end`.
> * Tests only pass if they return the symbol passed. That makes it
> harder for buggy tests to appear to pass when they actually never ran.
This is completely novel.
> * The assert macro uses simple heuristics to display the values that
> were passed to it. This makes it less necessary to have a family of
> assert macros for different purposes.
> * There is an assert-signals-condition macro to test that an
> expression causes a particular condition to be raised.
Similar to test-error in SRFI 64 and C/C.
> * Failure reports show the captured continuation of the failing test.
> This continuation can be used with MIT Scheme's debug to walk the
> stack of the failure, examining variables, etc. This is particularly
> useful when an unexpected condition is raised during the test.
This is seriously cool. Would be great to have this in a runner.
I think in addition to MIT Scheme, at least Gambit and Chez have a
continuation-aware interactive debugger.