Safe versus unsafe arrays
Bradley Lucier
(28 Jul 2023 21:52 UTC)
|
Re: Safe versus unsafe arrays
Arthur A. Gleckler
(28 Jul 2023 22:19 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(29 Jul 2023 05:58 UTC)
|
Re: Safe versus unsafe arrays
Arthur A. Gleckler
(29 Jul 2023 06:00 UTC)
|
Re: Safe versus unsafe arrays
Bradley Lucier
(13 Aug 2023 19:49 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(13 Aug 2023 19:59 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(29 Jul 2023 22:11 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(30 Jul 2023 07:41 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(30 Jul 2023 08:46 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(30 Jul 2023 09:00 UTC)
|
Re: Safe versus unsafe arrays
Bradley Lucier
(30 Jul 2023 20:39 UTC)
|
Re: Safe versus unsafe arrays Marc Nieper-Wißkirchen (30 Jul 2023 20:49 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(30 Jul 2023 21:37 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(31 Jul 2023 19:49 UTC)
|
Re: Safe versus unsafe arrays
Bradley Lucier
(12 Aug 2023 17:58 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(12 Aug 2023 18:09 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(12 Aug 2023 19:22 UTC)
|
PS: I would like to point out that using SRFI 145 is also a possibility. The idea of SRFI 145's "assume" is that it raises an exception if the assumption is not fulfilled when the compiler is not used with an unsafe optimization flag. In unsafe mode, the compiler may assume that it is fulfilled (which can be used by the compiler to infer types, bounds, etc.). An implementation safe by default of SRFI 231 could wrap all checks in SRFI 145 assumptions. After the program has been well-tested, the programmer can finally switch on the unsafe flag of its compiler. I would also like to add that what Bradley calls "call/cc-safety" is of a different type. The behaviour of conforming programs would change if the non-!-version were replaced with the !-version in unsafe mode. Am So., 30. Juli 2023 um 22:39 Uhr schrieb Bradley Lucier <xxxxxx@purdue.edu>: > > I'm going to reply to some comments, but may not get attributions > correct, sorry. > > I want to say at the start that the sample implementation uses the > following strategy: Compile the guts of all internal routines with > unsafe operations, and use wrappers to check arguments before calling > the unsafe routines. This minimizes the amount of error checking when > reusing code internally. > > Additionally, I just changed the version of SRFI 231 shipped with Gambit > to check all specialized array setter/getter arguments no matter whether > an array is "safe" or "unsafe". (This is allowed by the SRFI document.) > As of now, the SRFI 231 sample implementation still has the (default) > option of "unsafe" specialized array operations that can lead to crashes. > > John Cowan (lightly edited): > ============================================ > I think that overstates the problem. If we have a 2 x 2 unsafe matrix > M, then the storage object is a 4-element vector V, and so an attempt to > access M[2, 0] will try to access a non-existent element of V and raise > an exception, unless indeed the storage classes themselves are unsafe. > ============================================ > All storage class operations are compiled to unsafe code in the sample > implementation, so if an array has unsafe getters or setters, yes, the > system can crash. > > The matrix elements in your example could just be 4 strategically placed > memory slots in a larger body, so we don't know that accessing M[2,0] > will access a nonexistent element of the backing store or will raise an > exception. > > Marc Nieper-Wißkirchen: > ============================================ > Why should the Scheme system check the bound of V when asked for a > fast, unsafe version? Accessing matrix elements could be compiled > into a simple memory fetch. > ============================================ > The sample implementation's code to move array elements from one array > to another checks all array bounds and the type of elements *where > needed*, even for unsafe arrays. I felt that allowing crashes so deep > inside the library (this "move elements" code is used all over the > place) would just be impossible to debug. > > On the other hand, these checks do not preclude using memcpy when > circumstances allow, i.e., "a simple memory fetch", even when moving > elements to and from "safe" arrays. Additionally, no checks on array > values are performed when copying a u8 array to a s16 array, for > example, even if memcpy can't be used in this case. > > Further thoughts: > > (A) SRFI 231 abstracts arrays over many attributes: > > 1. layout of specialized array elements in memory. > > 2. types of specialized array elements (when not using > generic-storage-class) > > 3. generalized arrays versus specialized arrays. > > This abstraction imposes a certain general overhead. > > Making code "call/cc safe" imposes around a 50% time penalty in some > operations (and sometimes a much bigger space penalty), although this > penalty can be avoided with the "!" versions of array-copy, etc. And > now if people use the "!" procedures in a non-call/cc-safe way, which > "is an error", the sample implementation doesn't crash (I don't think). > > The worst execution time penalty I've seen so far using safe instead of > unsafe arrays is about 40%. Perhaps there are worse examples, but I > doubt that it's more than 100%. Perhaps the underlying abstraction > overhead obscures this "extra" overhead for safety. > > (B) I can imagine an implementation using JITted loops to make array-map > combined with %%move-array-elements run very fast, in the same way that > using memcpy where possible speeds simple array element movement by a > factor of 20 or more. > > (C) I would like people to use SRFI 231, I think it has a clean, > orthogonal design that incorporates much of the base functionality of > APL, NumPY, etc., and in a few ways it's more complete. But people > might decide not to use it for many different reasons. > > People may not use it because they find it too slow. > > But, slower than what? Julia? PyTorch? In that case, maybe Scheme and > this SRFI are not for those applications. > > But Gambit is a pretty fast Scheme implementation. Code that might have > 100% overhead on Gambit to make it both call/cc safe and getter/setter > safe may still run faster than non-call/cc-safe or > non-getter/setter-safe code on some other implementations. > > Or, people may not use this SRFI because, in the quest for speed, they > use (the default) "unsafe" specialized array operations that crash the > system when programming errors arise. It makes the library look buggy > and incomplete. > > I'm beginning to think the non-crashing property is more important than > a possibly Quixotic quest for "fast" array operations that may not turn > out to be very fast at all in an absolute sense. (Which reminds me of > the "sufficiently smart compiler" argument in some ways: > https://wiki.c2.com/?SufficientlySmartCompiler .) > > Brad