On Sun, 3 Apr 2005, Per Bothner wrote:

> As discussed in past messages, I'm leaning towords removing the
> concept of a test-result "value", which in the current specification
> is an association list.  Instead, I think it is better to make the
> "most recent test-result" be part of the test-runner, and allow
> extra properties there.
>
> Specifically, we add:
>
> ;; Get a named property - #f if unknown.
> (test-result [runner] 'property-name)
> These are properties of the current or most recent test.
>
> Exampes:
> (test-result 'kind) -> one of 'pass, 'xpass, 'fail, 'xfail, or 'skip.
> (test-passed?) -> same as (memq (test-result 'kind) '(pass xpass))
> (test-result 'source-line) -> line number if known or #f otherwise
> (test-result 'source-filename) -> file name known or #f otherwise
> (test-result 'source-form) -> test expresion as a list or #f
> (test-result 'test-name) -> test name or #f
>
> (test-result-set! [runner] 'property-name value) ;; set property
>
> (test-result-clear! [runner])
> This is called internally at the start of a test to reset the
> test-result state.  Specifically, all properties set by
> test-result-set! are cleared.
>
> The function on-test in (test-runner-on-test! runner on-test)
> is now a 1-argument function that only takes a test-runner.
> It gets the "test-result" from the test-runner.
>
> Note that both test-cases and test-runner can examine these
> properties.  For example (test-passed?) can be used to decide
> whether to skip subsequent tests.
>
> I also thing it simplies the implementation and api.  We can
> set properties at various statges during the execution of a test.
>
> Comments?  Does this seem like an improvement?

I like the idea of being able to programmatically decide wether to
continue, for example.  Then you could, e.g., define a custom runner that,
say, tolerates at most 10 errors and then skips everything else in the
rest of the group.

In some ways it seems better, although I don't quite get why you want to
*remove* test-result objects.  My mental model of test suites is that it's
a tree whose root is the suite, whose interior nodes are groups, and whose
leaves are test cases (each test case being represented by a test result).
I guess you have the same basic model, but none of those objects are first
class :-)

So I'd probably go the other way and make test results (or, more likely,
test cases) first class, something like so:

  (test-case [runner])  ->  current or most recent test case, or #f if none

  (test-case-kind tcase) -> one of (pass xpass ...) [*]
  (test-case-passed? tcase) ; same as (memq (test-case-kind tcase) '(pass xpass))
  (test-case-property tcase 'property [value])   ; property setter/getter
  (test-case-name tcase) -> test name

[* although 'kind' is looking like a strange name here; maybe
"disposition"?]

--
-- Donovan Kolbly                    (  xxxxxx@rscheme.org
				     (  http://www.rscheme.org/~donovan/