Safe versus unsafe arrays
Bradley Lucier
(28 Jul 2023 21:52 UTC)
|
Re: Safe versus unsafe arrays
Arthur A. Gleckler
(28 Jul 2023 22:19 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(29 Jul 2023 05:58 UTC)
|
Re: Safe versus unsafe arrays
Arthur A. Gleckler
(29 Jul 2023 06:00 UTC)
|
Re: Safe versus unsafe arrays
Bradley Lucier
(13 Aug 2023 19:49 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(13 Aug 2023 19:59 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(29 Jul 2023 22:11 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(30 Jul 2023 07:41 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(30 Jul 2023 08:46 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(30 Jul 2023 09:00 UTC)
|
Re: Safe versus unsafe arrays Bradley Lucier (30 Jul 2023 20:39 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(30 Jul 2023 20:49 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(30 Jul 2023 21:37 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(31 Jul 2023 19:49 UTC)
|
Re: Safe versus unsafe arrays
Bradley Lucier
(12 Aug 2023 17:58 UTC)
|
Re: Safe versus unsafe arrays
Marc Nieper-Wißkirchen
(12 Aug 2023 18:09 UTC)
|
Re: Safe versus unsafe arrays
John Cowan
(12 Aug 2023 19:22 UTC)
|
I'm going to reply to some comments, but may not get attributions correct, sorry. I want to say at the start that the sample implementation uses the following strategy: Compile the guts of all internal routines with unsafe operations, and use wrappers to check arguments before calling the unsafe routines. This minimizes the amount of error checking when reusing code internally. Additionally, I just changed the version of SRFI 231 shipped with Gambit to check all specialized array setter/getter arguments no matter whether an array is "safe" or "unsafe". (This is allowed by the SRFI document.) As of now, the SRFI 231 sample implementation still has the (default) option of "unsafe" specialized array operations that can lead to crashes. John Cowan (lightly edited): ============================================ I think that overstates the problem. If we have a 2 x 2 unsafe matrix M, then the storage object is a 4-element vector V, and so an attempt to access M[2, 0] will try to access a non-existent element of V and raise an exception, unless indeed the storage classes themselves are unsafe. ============================================ All storage class operations are compiled to unsafe code in the sample implementation, so if an array has unsafe getters or setters, yes, the system can crash. The matrix elements in your example could just be 4 strategically placed memory slots in a larger body, so we don't know that accessing M[2,0] will access a nonexistent element of the backing store or will raise an exception. Marc Nieper-Wißkirchen: ============================================ Why should the Scheme system check the bound of V when asked for a fast, unsafe version? Accessing matrix elements could be compiled into a simple memory fetch. ============================================ The sample implementation's code to move array elements from one array to another checks all array bounds and the type of elements *where needed*, even for unsafe arrays. I felt that allowing crashes so deep inside the library (this "move elements" code is used all over the place) would just be impossible to debug. On the other hand, these checks do not preclude using memcpy when circumstances allow, i.e., "a simple memory fetch", even when moving elements to and from "safe" arrays. Additionally, no checks on array values are performed when copying a u8 array to a s16 array, for example, even if memcpy can't be used in this case. Further thoughts: (A) SRFI 231 abstracts arrays over many attributes: 1. layout of specialized array elements in memory. 2. types of specialized array elements (when not using generic-storage-class) 3. generalized arrays versus specialized arrays. This abstraction imposes a certain general overhead. Making code "call/cc safe" imposes around a 50% time penalty in some operations (and sometimes a much bigger space penalty), although this penalty can be avoided with the "!" versions of array-copy, etc. And now if people use the "!" procedures in a non-call/cc-safe way, which "is an error", the sample implementation doesn't crash (I don't think). The worst execution time penalty I've seen so far using safe instead of unsafe arrays is about 40%. Perhaps there are worse examples, but I doubt that it's more than 100%. Perhaps the underlying abstraction overhead obscures this "extra" overhead for safety. (B) I can imagine an implementation using JITted loops to make array-map combined with %%move-array-elements run very fast, in the same way that using memcpy where possible speeds simple array element movement by a factor of 20 or more. (C) I would like people to use SRFI 231, I think it has a clean, orthogonal design that incorporates much of the base functionality of APL, NumPY, etc., and in a few ways it's more complete. But people might decide not to use it for many different reasons. People may not use it because they find it too slow. But, slower than what? Julia? PyTorch? In that case, maybe Scheme and this SRFI are not for those applications. But Gambit is a pretty fast Scheme implementation. Code that might have 100% overhead on Gambit to make it both call/cc safe and getter/setter safe may still run faster than non-call/cc-safe or non-getter/setter-safe code on some other implementations. Or, people may not use this SRFI because, in the quest for speed, they use (the default) "unsafe" specialized array operations that crash the system when programming errors arise. It makes the library look buggy and incomplete. I'm beginning to think the non-crashing property is more important than a possibly Quixotic quest for "fast" array operations that may not turn out to be very fast at all in an absolute sense. (Which reminds me of the "sufficiently smart compiler" argument in some ways: https://wiki.c2.com/?SufficientlySmartCompiler .) Brad