login signup
Further issues Wolfgang Corcoran-Mathe (20 Jun 2026 17:55 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (23 Jun 2026 12:08 UTC)
Re: Further issues and New Portable Implementation Wolfgang Corcoran-Mathe (23 Jun 2026 17:22 UTC)
Re: Further issues and New Portable Implementation Wolfgang Corcoran-Mathe (23 Jun 2026 20:00 UTC)
Re: Further issues and New Portable Implementation John Cowan (24 Jun 2026 01:42 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (24 Jun 2026 04:52 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (24 Jun 2026 04:28 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (24 Jun 2026 04:26 UTC)
Re: Further issues and New Portable Implementation John Cowan (24 Jun 2026 05:30 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (28 Jun 2026 06:25 UTC)
Re: Further issues and New Portable Implementation John Cowan (28 Jun 2026 10:22 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (29 Jun 2026 01:37 UTC)
Re: Further issues and New Portable Implementation Wolfgang Corcoran-Mathe (24 Jun 2026 15:56 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin (28 Jun 2026 06:11 UTC)
Re: Further issues and New Portable Implementation Andrew Tropin 23 Jun 2026 12:07 UTC

Show/hide attachments 
On 2026-06-20 13:55, Wolfgang Corcoran-Mathe wrote:

> I've started sketching a portable implementation of SRFI 269.  In the
> process, I've discovered a few more issues with the spec:
>
>
> 1. For some reason, 'metadata' appears as a quoted identifer in the
> definition of 'test' and its examples.  The form should be:
>
>     (test (<description> <context-id>) metadata <metadata>
>       <body>)

originally it was #:metadata, but later Ramin pointed out that it's too
guile-specific, we changed it to just 'metadata. Before the previous
draft the form was (test <description> 'metadata <metadata> <body>).

There were a few other ways to implement syntax for metadata and all has
their own pros and cons.  After a few iterations we ended up with this
one.

I can name a few good properties of it:
1. It's easy to omit metadata.

2. It's quite explicit (it's very unlikely that somebody would ever want
to have 'metadata as the first element of a test body.

3. It's consistent. suite, test, define-suite use the same way to add
metadata.

>
> (Also, why not put the metadata inside the list, after <context-id>?)

(test ("description" ctx ((fixtures db-connection-fixture
                                    init-db-users-fixtures)))
  <body>)

or

(test ("description"
       ctx
       ((fixtures db-connection-fixture init-db-users-fixtures)))
  <body>)

is arguabely less pretty than

(test ("description" ctx)
  'metadata
  '((fixtures db-connection-fixture init-db-users-fixtures))
  <body>)

When I skim tests code I'm interested to see the description, it's
inconvinient, when something tries to draw my attention.  I only
interested to see metadata, when I decided to dive into test.

>
>
> 2. context data: It's hard to use a value of a completely unspecified
> type.  I understand you want to leave this open to expansion, but
> I think the SRFI should establish a baseline.  Make it another alist,
> for example, and add the test/description and (if it exists) the
> suite/description keys.

This is a good idea, I added a task to add a recommendation on context
initial value.  I wrote a blog post on this topic, it provides overal
idea of how it should look like, but I still not completely settled yet.
I will add at least some base recommendations in the next SRFI revision.

https://trop.in/blog/context-makes-tests-reusable

>
> If you really want to leave it unspecified, you could still give some
> type-generic procedures / messages for getting data out of a context.
> (For example, a 'context->alist' procedure.)
>
>
> 3. Since tests "should not depend on side effects", I assume mutating
> the context data is forbidden.  But mutation of context could be a
> channel for passing data from a test back to the runner.  Should there
> be some non-mutative way to do this?
>

"should not depend on side effects" was here before we added a context
to specification.

By this I mean test code must not use any non-constant values from
lexical scope, e.g.

(suite "my suite"
  (let ((db (make-atomic-box '())))
   (test ("my test" ctx)
     (mutate-db-somehow! db)
     ...)
   ...))

There is no guarantee when and how many times test will be executed and
depending on a stateful variable (no matter lexically scopped or global)
will make the test execution result completely unpredictable. Or more
likely predictably screwed up :)

With the introduction of the context into specification, we can now fire
side effects, but strictly targeted to context values.  The context is
cleaned up and reconstructed on every test execution.

Both test runner and test can modify it.  I'm not sure yet in which case
we would like test to change it, but we definitely have power to do so:
provide an atomic-box to context and let test set it.

On the other hand the use case for test runner is quite clear: Providing
suites and tests metadata, or extending context value with fixtures.
Overall the context remains an immutable value and fixtures just create
a new context and pass it down the stack, however some of the value can
be "sideeffectful" like db connections, sockets or atomic-boxes and both
tests and other fixtures would be able to fire necessary side effects.

P.S. So to be more precise test has no ability to modify context as it
should be immutable, but can touch sideeffectful values from it.

>
> 4. "Tests that do not need the context can use _ as the identifier."
> I assume this means that 'test' should specifically omit binding the
> context value if the context identifer is free-identifier=? to _.  Or
> is this just a recommend convention?

Yeah, just a recommended convention.

>
> 5. A definition of "fixture value" would be helpful.  (Pardon my
> ignorance of testing jargon.)

Should be clear from the blog post, but yeah, definitely worth a
mention in SRFI.  Added a note for myself.

> 6. There seems to be no way to *run* a test or suite!  I searched in
> vain for runner/run-test and runner/run-suite messages.  The SRFI is
> quite clear that runner/load-test and runner/load-suite are "loading
> form[s]" that do not execute anything.

Good catch. You can take a look at suitbl library, it implements
`auto-run?` configuration option, which runs tests immediately after
they loaded, but also allows to send a separate message.

Probably, it's better to define a message type or provide some
recommendations on this regard in SRFI.  I wrote a not for myself for
the next revision.

>
> 7. Minor: Replace or define the nonstandard 'assoc-ref' procedure.

Right, added a task.

Thank you again for very valueable feedback.  All notes are written
down.  Let me know if you have further thoughts! :)

--
Best regards,
Andrew Tropin