On directory-fold: Gauche also has directory-fold, but it actually traverses the directory recursively.  Since the Chicken-type directory-fold is trivial to implement (especially, if we have generator interface, we can just call generator-fold), I'd rather like to see it not in srfi.

On open/read/close-directory - I second for having one (or similar).  A drawback of generator interface is that you don't have control of releasing the resource.  Sometimes you want to explicitly close the directory. (E.g. a long-running process partially traverses a directory; it's done with it but just leave closing the directory to its GC.  You try to unmount a volume and found that opened directory reference is kept by that process. Ouch!)

On Tue, Dec 10, 2019 at 4:00 PM <xxxxxx@ancell-ent.com> wrote:
Glad to hear that.  There's a stronger argument for open-/read-/close-directory, on Linux and OpenBSD (only ones I checked), they're library, not system calls.  Looking at the glibc code, it really and truly opens the directory as a file, and reads dirent structures (a bit indirectly, which I've not drilling down to, bit suspicious given all the filesystems Linux supports).  So failure to exhaust the generator would add to your limited count of open file descriptors ... hmm, do Scheme implementations tend to GC and retry if file descriptors are exhausted?  In any event, if the partial read of a directory is a valid use case, we need to leave these basic wrappers in.

If you're convinced of utility of the simplicity of directory-files, which along with make-directory-files-generator provide the ideal 80/20 or better solution, sounds like we're set.  Although Lassi mentioned directory fold; I use a modified for dot-files version from Chibi Scheme for directory-files:

;;> The fundamental directory iterator.  Applies \var{kons} to
;;> each filename in directory \var{dir} and the result of the
;;> previous application, beginning with \var{knil}.  With
;;> \var{kons} as \scheme{cons} and \var{knil} as \scheme{'()},
;;> equivalent to \scheme{directory-files}.

(define (directory-fold dir kons knil . o)
  (let-optionals o ((dot-files? #f))
    (let ((do (open-directory dir dot-files?)))
      (let lp ((res knil))
        (let ((file (read-directory do)))
          (if (not (eof-object? file))
              (lp (kons file res))
              (begin (close-directory do) res)))))))

Which is called with (directory-fold dir cons '() dot-files?).  I have no informed opinion on whether it should be added to the API.

- Harold

----- Original message -----
From: John Cowan <xxxxxx@ccil.org>
Date: Tuesday, December 10, 2019 6:08 PM

Good points. I'm convinced.

On Tue, Dec 10, 2019 at 6:44 PM <xxxxxx@ancell-ent.com> wrote:

From: John Cowan <xxxxxx@ccil.org>
Date: Tuesday, December 10, 2019 4:41 PM

The non-generator interfaces are scsh legacy, and I'm happy for them to be removed if nobody objects.  It's obviously trivial to produce the generator over the scsh interface.

Generators rule!

I like directory-files because it's the dead simple API: directory path in, list of entries out (dot-files optional).  It is of course redundant with the generator interface, and would not be the end of the world to omit.

I'd like much stronger arguments before removing open-/read-/close-directory, because on a POSIX system you do have to wrap those system calls to create the friendlier interfaces, so the cost of exposing them is small, and I'm leary of forcing all users to use a generator, no matter how nice they are.

For example, use cases which don't require reading the entire directory: maybe you need to check for a few files and can stop after you find them, or only need to confirm it has at least N files.  close-directory only happens when the generator is exhausted, or, if you get the underpinnings correct, upon it getting gc-ed.

And my general philosophy is direct, albeit smartened up and safer direct POSIX wrappings are the explicit, official target of this SRFI, with convenience procedures like make-directory-files-generator as bonus conveniences.  Omitting open-/read-/close-directory would be a major departure, only comparable to not providing a full stty interface because that's a lot of work no longer relevant for most uses today, and using the magic of dynamic-wind avoids most causes that leave the terminal in a messed up non-echoing mode, requiring blindly entering "^Jreset^J".

- Harold

----- Original message -----