Hey Daphne,
thank you very much for sharing your ideas with us! It looks like a very useful and usable concept. However, there are three or four issues why I don't think that cursors as proposed right now shall become the fundamental iteration abstraction:
(1) The cursor state has to be made explicit in a number of state variables. This is fine for simple iterations, but tedious when the state implicitly consists of values of a number of local variables. Think of a pure version of `make-coroutine-generator`, say `make-coroutine-cursor`. Your proposal is like coding coroutines in C ([1]) vs. modern concepts like coroutines in C++20.
(2) Cursors are not fully lazy. You cannot write a true `stream->cursor` without unpacking the stream first.
(3) The current proposal doesn't incorporate multiple values (but this can probably be corrected).
(4) Is the difference between cursors and streams enough to warrant a new concept? (You have stream-null?, stream-car, stream-cdr instead of cursor-end?, cursor-head, cursor-next, but that's it, mostly.)
I would like to recapitulate my iterator proposal. An iterator ITER is a procedure that when called on two arguments SUCCESS and FAILURE tail-calls FAILURE if the sequence represented by ITER is empty, and otherwise tail-calls SUCCESS with an iterator representing the rest of the sequence and the values making up the head of the sequence.
These iterators do not suffer from the points above. For example, a stream iterator would be:
(define (stream->iterator s)
(lambda (success failure)
(if (stream-null? s)
(failure)
(success (stream->iterator (stream-cdr s)) (stream-car s)))))
In particular, it is completely lazy and does not prematurely unpack the stream.
You can turn any cursor into an iterator by:
(define (cursor->iterator c)
(lambda (success failure)
(if (cursor-end? c)
(failure)
(success (cursor->iterator (cursor-next c)) (cursor-head c)))))
Modulo the issues above, you can do the reverse by:
(define (iterator->cursor iter)
(iter
(lambda (iter val)
(make-cursor
(lambda (next iter)
(iter
(lambda (iter val)
(next val iter))
(lambda ()
(next))))
val iter)
(lambda () (make-end-cursor))))
Here, one can clearly see that the current cursor proposal is not lazy enough for the general case.
What about the following cursor proposal instead (which is equivalent to iterators):
(define-record-type cursor
(%make-cursor promise)
cursor?
(promise %cursor-promise))
(define-record-type %cursor-state
(%make-cursor-state done? values next)
%cursor-state?
(done? %cursor-state-done?)
(values %cursor-state-values)
(next %cursor-state-next))
(define (make-cursor iter)
(%make-cursor (delay
(iter
(lambda (iter . val*) (%make-cursor-state #f val* iter))
(lambda val* (%make-cursor-state #t val* iter)))))
(%cursor-state-done? (force (%cursor-promise c)))
(define (cursor-head c)
(apply values (%cursor-state-values (force (%cursor-promise c)))))
(define (cursor-next c)
(let ((iter (%cursor-state-next (force (%cursor-promise c)))))
(%make-cursor (delay
(iter
(lambda (iter . val*) (%make-cursor-state #f val* iter))
(lambda val* (%make-cursor-state #t val* iter)))))))
Thanks to the use of delay/force, it also gets the corner cases (see the documentation for delay/force) right.
For iterators, a convenience syntax like the one in your proposal is not really needed:
(cursor name ((var init) ...) body)
would simply be
(let f ((var init) ...)
(lambda (success failure)
(let ((name (case-lambda
(() (failure))
((val . init*) (success (apply f init*) val)))))
body)))
As I consider ad-hoc overloading the "next" procedure depending on the number of arguments to do two very different things an error, I don't propose to use the latter iterator equivalent. As one can clearly see, it hardly generalizes when we also want to yield values through the failure continuation.