Hello Marc,

Thank you for your feedback.

My hope and intention is to create iterators which can be efficiently traversed without consing even on non-optimizing Scheme implementations, to fully replace SRFI 158. This means that neither producer nor consumer needs to allocate during traversal — your proposal requires an allocation on the producer side for every step of the iteration.

FWIW John Cowan asked on IRC if it’s really necessary that the same cursor be capable of being advanced both functionally and imperatively. I think not, and I may make it an error to call cursor-next on a cursor when cursor-next! has already been called, or vice versa. This would enable certain optimizations, albeit delay the decision of whether to use them until the first advancement. But I resist having two disjoint cursor types in a way that would require the writer of the producer to write two different procedures, one for functional and one for imperative traversal.

> On 2 Dec 2021, at 13:41, Marc Nieper-Wißkirchen <xxxxxx@gmail.com> wrote:
>
> Hey Daphne,
>
> thank you very much for sharing your ideas with us!  It looks like a very useful and usable concept.  However, there are three or four issues why I don't think that cursors as proposed right now shall become the fundamental iteration abstraction:
>
> (1) The cursor state has to be made explicit in a number of state variables.  This is fine for simple iterations, but tedious when the state implicitly consists of values of a number of local variables.  Think of a pure version of `make-coroutine-generator`, say `make-coroutine-cursor`.  Your proposal is like coding coroutines in C ([1]) vs. modern concepts like coroutines in C++20.

I think a make-coroutine-cursor could be provided fairly trivially on top of the current proposal for the basic mechanism. After all, generator->cursor is possible:

(define (generator->cursor gen)
  (cursor next (itm (gen))
    (if (eof-object? itm)
        (next)
        (next itm (gen)))))

> (2) Cursors are not fully lazy.  You cannot write a true `stream->cursor` without unpacking the stream first.

Hmm, I can foresee cases where the requirement to fill in the initial value at cursor creation time (rather than when the initial place is accessed) could be a problem. (The above generator->cursor is arguably such an example!) They could be made lazier, of course.

> (3) The current proposal doesn't incorporate multiple values (but this can probably be corrected).

Noted above. I’m not 100% certain this is truly useful. There’s the case of iterating over key-value mappings, but SRFI 158 doesn’t account for that, and none of the current key-value map libraries in R7RS Large provide generator conversions.

> (4) Is the difference between cursors and streams enough to warrant a new concept?  (You have stream-null?, stream-car, stream-cdr instead of cursor-end?, cursor-head, cursor-next, but that's it, mostly.)

The possibility for imperative, non-consing traversal is the main advantage of cursors over SRFI 42 streams. Perhaps that can be hacked into lazy streams, but it would not be pretty. (Of course, ‘pretty’ is relative when we’re talking about mutating state anyway ;-).)

> As I consider ad-hoc overloading the "next" procedure depending on the number of arguments to do two very different things an error, I don't propose to use the latter iterator equivalent. As one can clearly see, it hardly generalizes when we also want to yield values through the failure continuation.

I agree overloading the next-proc like this is not the most elegant. Perhaps I will fix that to use two different procedures for continuing and terminating the iteration, as you suggest.

> Marc

Daphne