I did not consider the stream-ext library in the design of SRFI-41.  In general, as explained in the SRFI, I designed SRFI-41 according to the advice of Antoine de Saint-Exupéry: "Il semble que la perfection soit atteinte non quand il n’y a plus rien à ajouter, mais quand il n’y a plus rien à retrancher."  (“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”)  And I think I got a couple of things wrong -- if I was to rewrite SRFI-41 today, I would take away even more.

Streams are not lists.  Scheme ensures there are substantial disadvantages to using streams rather than lists, due to the underlying promises that require numerous type conversions, so streams should be used only when the sequence of elements is truly infinite (such as mathematical series) or when there is some clear advantage of laziness (such as reducing the number of passes through a large data set).  Writing a library that duplicates SRFI-1 for streams seems to me to be folly.

I looked briefly at your list of suggested extensions.  A few may be useful for a modest extension library, if you have in mind a particular set of uses --  some of the examples in the SRFI, such as stream-partition, fall into this category.  Some certainly don't belong in a general-purpose library -- if you need symbol->stream to convert the name of a symbol into a stream of characters, you can write it as part of your program.  Many -- such as stream-butlast -- make sense only for lists (which are materialized in their entirety) and not for streams (which may never be materialized).  I find it amusing that you consider stream->vector and vector->stream, since the pure FP people have such trouble with mutable arrays.  And even the design is poor -- stream-butlast-n should almost certainly be merged into stream-butlast with an optional argument, but stream-reverse shouldn't also optionally append a list (or is it a stream?).

You say that it is so obvious that all these functions should be included that you won't waste your time justifying it, but to me it is anything but obvious -- in fact, it is obvious to me that some of the functions you mentioned should never exist in the context of streams implemented for Scheme, much less be included in a standard library.  On the other hand, in the context of streams implemented for Haskell, it may make sense to include some of the list-oriented functions, since Haskell has no finite-list type and uses streams in its place.  Remember: streams are not lists.

You are of course free to use SRFI-41 as the basis for an extended library of stream functions.  I ask only that you do not call it streams, or streams-primitive, or streams-derived, so as not to cause confusion with the SRFI.  I also recommend that you switch over as soon as possible from the deprecated SRFI-40 to SRFI-41.

By the way, you should have mentioned as you critiqued SRFI-41 that you are the author of stream-ext.

Phil

On Tue, May 5, 2009 at 9:16 PM, Alejandro Forero Cuervo <xxxxxx@freaks-unidos.net> wrote:
> SRFI-40 is deprecated due to a memory leak.  Please port SRFI-41 instead,
> and adapt any code that uses SRFI-40 to the new interface.

Did you, during the design of srfi-41, consider the existance of the
stream-ext library, which implements a lot of stream functions on top
of srfi-40?

http://chicken.wiki.br/eggref/3/stream-ext

It's first version was released a long time before srfi-41 and only
small changes have been made since then.  The naming conventions and
the semantics match those of srfi-1 as close as possible, to make
things as consistent as possible.

I've used the stream-ext library in many applications that represent
strings as streams of characters throughout.  The largest of these is
probably Svnwiki.

I'm worried about the incompatibilities I see between stream-ext and
srfi-41.  My concerns are the following:

1. srfi-41 provides a very small subset of the functionality that I
think that any program that uses streams significantly will need.  As
such, I think it fails significantly at providing "syntax derived from
those primitives that permits convenient expression of stream
operations".  Most of the functions I list below can be implemented in
a portable manner, as stream-ext does.  To be fair, srfi-41 does seem
to provide some functions that stream-ext does not.

2. More importantly (the previous concern can be solved with an
additional library), a small portion of the interface exported by
srfi-41 differs from that in the stream-ext library.  I provide some
cases below and explain why I believe the semantics from stream-ext
are slightly preferable, mostly because of consistency with the srfi-1
list-based counterparts.  I quite like the interface of srfi-1 and I
find that, by providing inconsistent counterparts, srfi-41 is making
it slightly harder to use streams than stream-ext.

I may be the person who has written the most Scheme code using srfi-40
streams.  I've put special care into the design of the interface of
the stream-ext library.  This interest I have in using streams in
Scheme makes the fact that srfi-41 offers an inferior interface rather
frustrating for me.

Now onto the differences and similarities:

The following is a list of symbols provided by stream-ext and srfi-41
with apparently the exact same semantics:

 list->stream
 stream-ref
 stream-length
 stream-append
 stream-take-while
 stream-drop-while

The following is a list of symbols available in stream-ext but not in
srfi-41, which I believe most software using streams would benefit
from:

 stream-xcons
 stream-cons*
 stream-tabulate
 stream-iota
 make-infinite-stream

   I think these should be included as counterparts to the srfi-1
   versions for lists.  The make-infinite-stream should probably be
   added (see stream-ext's make-stream, discussed bellow).

 stream->string
 string->stream
 stream-downcase, stream-upcase
 stream-lines
 stream-unlines

   I think these should be included: if one does include
   port->stream, encouraging the use of streams of characters to
   represent ports/streams, why not go the extra mile of simplifying
   conversion to and from strings and handling of streams of
   characters?

 stream->vector
 vector->stream
 number->stream
 stream->number
 stream->symbol
 symbol->stream

   And if we include the stream->string and string->stream symbols,
   encouraging the programmer to use streams throughout his program
   to represent strings, these should probably also be included,
   specially the vector ones.  I don't think we should force all
   programs that use streams to define this compositions directly.

 iterator->stream

   This is a fundamental block for turning iterators into streams.
   I've found it extremely useful in my programs that use streams.
   For example, list->stream can be trivially implemented as:

     (define (list->stream l)
       (iterator->stream
         (lambda (return stop)
           (for-each return l))))

   Among *many* other things, this makes it trivial to do things like
   'with-output-to-stream', given the right support from the
   implementation (for creating special ports).  Also,
   iterator->stream does not depend on anything non-portable, simply
   on call/cc.

 write-stream

   The counter-part to port->stream.  Most of my programs have a lot
   of functions that generate streams (see for example my html-stream
   library).  A lot of these will be wrapped in a simple call to
   write-stream, to output these to the right port.  I think it
   should be included in a general purpose streams library.

 stream=
 stream-prefix=
 stream>, stream<
 stream-caar ... stream-cddddr
 stream-first ... stream-tenth
 stream-intersperse
 stream-split
 stream-last
 stream-last-n
 stream-butlast
 stream-butlast-n
 stream-length>=
 stream-count
 stream-partition
 stream-remove
 stream-sort
 stream-find
 stream-find-tail
 stream-any
 stream-every

   I'm using all these quite often.  I feel all of them deserve
   inclussion in a general purpose for construction of streams.  I
   think this is obvious enough that I won't waste my time justifying
   it.  If you think a given one of these should not be included, ask
   me and I'll explain why I think it should.

   I find it a bit surprising that stream-partition, stream-remove,
   stream-find and stream-sort were not included in the library, even
   though they were considered general enough that they were defined
   as examples (though stream-sort is implemented using merge-sort).

 with-output-to-stream

   Very convenient for turning code that generates output to its
   current-output-port into code that builds a (lazy, obviously)
   stream.  However, I suppose this can't be implemented in a
   portable manner. :-/

 with-input-from-stream

   The rationale is very similar to that of with-output-to-stream.
   However, I've found this procedure a bit less useful than the
   other.

The following is a list of symbols provided by stream-ext with
semantics incompatible with those of srfi-41.  I believe the
definition in stream-ext is more adequate for the reasons I explain.

 make-stream

   stream-ext's definition matches the counterpart from srfi-1's
   make-list.  srfi-41 defines this but, fortunately, does not export
   it.

 stream->list

   I don't like the idea of the optional number-of-elements
   arguments.  This should simply work only on finite streams (and
   one should use stream-take for infinite streams).  I think
   stream-ext's definition is more adequate as it more commonly
   reflects the expectation from the programmer (of just converting a
   stream to its corresponding list); taking the head of the list
   should not be done by this function.  I also tend to dislike
   optional parameters preceeding mandatory parameters.

 port->stream

   The interface described in srfi-41 is a subset of that in
   stream-ext: stream-ext adds two optional parameters that make this
   procedure a lot more flexible/usable, without any disadvantages.

 stream-take, stream-take-safe
 stream-drop, stream-drop-safe

   These are *almost* compatible.  The only difference is that, in
   consistence with srfi-1, I've decided to make it an error to take
   more elements than the stream has.  srfi-41, instead, has decided
   to not make it an error.  I prefer the stream-ext behavior simply
   for being consistent with what the srfi-1 counterparts do.  I want
   to make the streams and lists libraries as compatible as possible.

 stream-concatenate

   srfi-41 decided to call this stream-concat, which is inconsistent
   with srfi-1.  I prefer stream-ext's consistent naming convention.

 stream-reverse

   This is *almost* compatible, but stream-ext allows the caller to
   pass a list to be appended to the result, which I think makes the
   procedure slightly more useful.

The following are other symbols in stream-ext that I haven't had time
to compare with those in srfi-41:

 stream-fold
 stream-fold-right
 stream-fold-right-delay
 stream-span, stream-break
 stream-index
 stream-member, stream-memq, stream-memv
 stream-format
 stream-delete
 stream-delete-duplicates

I suspect the fold procedures provided by stream-ext are far more
usable than those in srfi-41, but, as I said, I haven't had time to
compare them.

The following are symbols in srfi-41 and not in srfi-40 that I haven't
had time to compare with those in stream-ext:

 stream-lambda
 define-stream
 stream-constant
 stream-fold
 stream-from
 stream-iterate
 stream-let
 stream-match
 stream-of
 stream-range
 stream-scan
 stream-unfold
 stream-unfolds
 stream-zip

Thanks.

Alejo.
http://azul.freaks-unidos.net/