My experience with the Common Lisp Pathname fiasco (and supporting an abstraction layer for them over 17 active and semi-active CL implementations on five extremely different OSes — Unix, Windows, Genera, MacOS 9, Mezzano) taught me that: - The wrong, underspecified and leaky abstraction is actually extremely costly to work with. You end up having to code for M*N combinations of (language implementation and operating system) each having reinvented a square wheel, rather than M+N adapters. - The goal of a standard like a SRFI should be to minimize the work to O(M+N) instead of O(M*N), and interoperation to O(M+N) still instead O(A**(M*N)). The only solution that makes sense then is: (1) Specify a portable FFI between Scheme and each of the underlying environments that define APIs: Windows, macOS, Linux, BSD, Android, iOS, the JVM, Genera, etc. Small, finite amount of convention to be agreed upon. (2) Then, any system-provided API can be accessed through in Scheme through a standard 1-to-1 mapping. No need for lengthy arguments, all the hard work is delegated to whichever body defines those APIs. Maybe the mapping comes in a couple of variants, but this can be kept style-specific, instead of implementation-specific. Your mutable vs linear variants will be universally available everywhere. (3) High-level features can be implemented purely as portable user-level libraries. No need for lengthy arguments here, since there is no need anymore for tens of people agreeing on the same thing for it to be widely and wildely useful. Then, there remains only one difficulty for those who'd like to implement one-current-directory-per-thread: virtualization of filesystem access. For that you need: (4) A lot of work replacing all those open by openat, unlink by unlinkat, foo by fooat, etc. But it only needs be done one, and will then magically work on all implementations running on a sufficiently advanced Unix OS. Some mock up for Windows might be possible, or not, but once again would only need be written once. (5) Some reflection mechanism to virtualize those system calls *including as used by the underlying compiler*, so you can compile on your virtual filesystem, intercept console access, track which files were touched in your caching build system, do your own user-level strace, ptrace, access-control, accounting, monitoring, performance-measurement, caching, scrubbing, frobbing, alerting, etc. That's the kind of language mechanism that requires standardization and could be tremendously beneficial and should be the topic of a SRFI. Doing (4) and (5) separately this way is MUCH LESS WORK, and MUCH LESS ARGUMENT for MUCH MORE BENEFIT than endlessly arguing about a SRFI for thread-local current-directory or not. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Memetics isn't about the "truth" of memes, but about their stability. On Thu, Apr 30, 2020 at 3:27 PM Shiro Kawai <xxxxxx@gmail.com> wrote: > > I share Faré's concern. > It is ture that process-wide chdir can't be safely used in MT environment. > On the other hand, in which occasion you want to change the working directory? > > Aside from the case that the daemon program wants to sit on a specific directory (rather than the directory > where the daemon is invoked), the only case I can think of is in interactive REPL session (much like you type 'cd' in the shell), > and depending the application you want it to be contained in the thread that handling REPL, or > you want to change the actual process working directory; you may want the change to be temporarily, > or stick after the REPL session. Trying to identify all possible cases of such a specific application > in "POSIX srfi" seems out of place to me. > > It is certainly difficult to implement REPL-local chdir (which won't affect other running threads) without having > working-directory API to which *all* pathname handling procedures honor. But we'd better discuss it separately > from POSIX interface, concentrating on various use cases and possible implications. > > For POSIX srfi, I'd be happy to say that we have interface to POSIX getcwd() and chdir(), and yes, chdir() is dangerous > in MT application, just like as any other POSIX applications, don't use it unless you know what you're doing, > you've been warned, period. > > > > > On Thu, Apr 30, 2020 at 8:26 AM Lassi Kortela <xxxxxx@lassi.io> wrote: >> >> > Sorry for being a late-comer to this discussion, but I *staunchly* >> > oppose the entire "provide a thread-local current-directory API when >> > the system doesn't" approach: >> >> Thank you for commenting. >> >> > - The standard should provide as thin layers around the operating >> > system as makes sense, and let users build their abstractions. >> >> It's not yet clear whether this is going into standard Scheme >> (R7RS-large) or staying as an individual SRFI. >> >> > Either the abstractions can be built, in which case we should let the user >> > build them. >> >> The logical conclusion is that the language would end up with nothing >> save for `lambda` and friends. >> >> For example, SRFI 1 (List library) could be built from simpler parts but >> there would be no sense in everyone reinventing the wheel. >> >> > Or they cannot be built, in which case the radioactive >> > mess of failure trying to build them should not be part of the >> > language. >> >> If leaky abstractions may never be built, we need to give up on Scheme >> and go build with assembler or logic gates or something. A measure of >> pragmatism is required to deal with software (or indeed, life itself). >> >> > - In particular, if the language does not allow the abstraction to be >> > built *as a library*, then the problem comes with the language's lack >> > of abstraction, not with its lack of primitives. >> >> Isn't the big idea of Lisp to blur the distinction between languages and >> libraries? A big Lisp dialect is a small core plus lots of useful >> libraries. Even core Haskell looks like typed Lisp so arguably the >> syntactic sugar is library code. >> >> Mending Scheme's core abstractions to conform to Posix sounds like the >> language design equivalent of what they call overfitting in ML. Lisp has >> survived for decades because aversion to overfitting is a fundamental >> tenet of Lisp culture. >> >> > - If the operating system interface is completely broken (such as >> > inotify being unreliable or signals being lost), no amount of papering >> > over it can save the user, let alone provide them with a >> > one-size-fits-all solution with dealing with the issue. >> >> Agreed. There have been problems with previous attempts to match Scheme >> ports 1:1 to Unix file descriptors, maintaining the correspondence in >> all circumstances. It's very hard and still doesn't really work. >> > - POSIX says that threads share current directory. RnRS should >> > acknowledge that, and anything else is bullshit. Don't break user >> > expectations with "magic" that is actually a curse. >> >> That's the POSIX current directory. It has been argued extensively on >> the SRFI 170 mailing list (especially by Marc from Gambit and Per from >> Kawa) that the notion is troublesome for multi-threaded Scheme code. >> >> > - Using directory names is STUPID AND EVIL and chdir should NEVER, >> > EVER be used outside the user control. If you *HAVE* to do something, >> > use fchdir instead. There are ESSENTIAL CASES where the entire >> > "normalize to absolute path" makes NO SENSE WHATSOEVER: the process >> > may not have access to parent directories, hidden by a mount, by a >> > chroot, by a missing a unlink/unlinkat, by a missing permission or >> > capability, a renamed directory, a moved symlink, etc. Trying to >> > change directory by name is then completely broken and a huge security >> > vulnerability. Also, not all systems that you may run Scheme on may >> > have a functional fchdir. >> >> Here you raise good points. Opening the current directory and using >> fchdir() in the parameter converter may make sense. As you say, it may >> still break and will certainly be non-ideal. But it's not clear that >> Unix affords an ideal solution no matter what we do.
