TL;DR summary:
With reasoning given below, I've tentatively decided that:
(a) When implicitly broadcasting generalized arrays as array arguments
to other procedures, the library will first array-copy the generalized
array to a specialized array and then broadcast the specialized array.
There is some precedent for this in the sample implementation, wherein
the "call/cc-safe" versions of routines that splice together existing
array arguments in various ways to get a result array, copy all their
generalized array arguments to specialized arrays before assigning any
elements to the result array. (And these conversions may occur multiple
times if any of the array getters play around with call/cc.)
There will be a procedure, array-broadcast, to explicitly broadcast
arrays, which will return a generalized array result if given a
generalized array argument, so if you don't want the library to
automatically copy a generalized array argument before implicit
broadcasting, array-broadcast the generalized array argument to the
broadcast domain explicitly.
(b) There will be the following procedures:
(array-broadcast array new-domain)
where new-domain is compatible with the domain of array, that just
manipulates getter/setter multi-index arguments and applies to
generalized arrays without first copying to a specialized array;
(compute-broadcast-interval intervals)
which checks that the intervals in the list intervals are compatible for
broadcasting and returns the broadcast interval;
(object->array obj)
which wraps any Scheme object in a zero-dimensional array, thus making
it available for broadcasting;
(array-insert-axis array k)
(interval-insert-axis interval k)
which insert a new k'th axis with lower bound 0 and upper bound 1 into
array or interval. array-insert-axis manipulates the multi-index
arguments of the array without copying.
*** Background ***
I'm been studying array broadcasting in Racket's math/array library.
Not so much as a direct guide, but to understand how another largish
array library in Scheme (well, Typed Racket) does things. math/array
has implicit array broadcasting, like Python's NumPy array library.
I've also talked to some graduate students who do scientific computing
with NumPy arrays, and some of them use implicit broadcasting so
naturally they don't recognize they're doing it before it's pointed out
to them.
So I guess it's important to have implicit array broadcasting in a
Scheme array library.
I've found it difficult to think about the interaction of implicit array
broadcasting and generalized arrays, whose getters could have side
effects (like reading from a file) or could capture continuations, which
could later be re-invoked. I felt confused, there is a discussion here
on the Racket Discourse:
https://racket.discourse.group/t/broadcasting-mutability-and-non-strict-arrays-in-math-array/3956/
where Philip McGrath offers some perspective as a user of math/array.
*** implicit array broadcasting ***
The following SRFI 231 procedures are candidates for implicit
broadcasting of array arguments:
Procedure: array-map f array . arrays
Procedure: array-for-each f array . arrays
Procedure: array-fold-left operator identity array . arrays
Procedure: array-fold-right operator identity array . arrays
Procedure: array-any predicate array . arrays
Procedure: array-every predicate array . arrays
Procedure: array-stack k arrays [ storage-class [ mutable? ] ]
Procedure: array-stack! k arrays [ storage-class [ mutable? ] ]
Procedure: array-append k arrays [ storage-class [ mutable? ] ]
Procedure: array-append! k arrays [ storage-class [ mutable? ] ]
array-map is described in the SRFI document as "set[ting] up operations
to be executed in the future".
If, however, array arguments are broadcast so as to to set up the
generalized array result of array-map, that broadcasting can be
considered as some operations to be done now to prepare arrays to be
mapped later. Copying generalized array arguments that are to be
broadcast to match the dimensions of other arguments just adds a bit
more processing to prepare the array arguments for computing the result
of array-map
The array arguments of array-map, ..., array-every don't have any
restrictions on the storage-classes of the array arguments, so any
generalized array to be broadcast should be copied to a specialized
array with generic-storage-class before broadcasting.
I've separated array-append and array-append! from the other procedures
because when appending arrays along the kth axis, broadcasting, if used,
is applied to argument arrays along all axes except the kth.
*** array-broadcasting parameter ***
Racket's math/array defines a parameter array-broadcasting that controls
implicit broadcasting of argument arrays. It can take the values #t,
#f, and permissive. I think having a parameter that can take the values
#t and #f is a good idea---passing arrays of different shapes to a
procedure may in fact be an error, and automatically broadcasting the
arrays may not always be helpful.
In Racket's math/array, the parameter value "permissive" operates in the
following way.
Normally, the kth axis is eligible for broadcasting if its lower bound
is 0 (which is always true in math/array) and upper bound 1, and
(array-ref broadcasted-A ... i_k ...)
==>
(array-ref original-A ... 0 ...)
with a 0 as the kth index of the original array.
When the parameter array-broadcasting is 'permissive, then the upper
bound, u_k, of the kth axis is allowed to be greater than 1, and for all
i_k we have
(array-ref broadcasted-A ... i_k ...)
==>
(array-ref original-A ... (modulo i_k u_k) ...)
Note that (modulo i_k 1) => 0 for all i_k, so this is indeed a
generalization of the usual broadcasting rules for axes with upper
bounds 1. The math/array documentation notes that this is allowed in R.
Because "permissive" broadcasting is not an affine mapping of the array
indices I don't want to allow it in this library.
Aside: I'm restricting array index manipulations to affine mappings
because I believe that such a restriction allows fast implementations.
I'm not going to claim the sample implementation is "fast", but I
believe that fast implementations, along the lines of PetaLisp, can
exist with this design.
*** explicit array broadcasting ***
Racket's math/array has (using the Typed Racket notation, about which I
know very little):
(array-shape-broadcast dss [broadcasting]) → Indexes
dss : (Listof Indexes)
broadcasting : (U Boolean 'permissive) = (array-broadcasting)
The Indexes type is a vector of nonnegative integers that are possible
lengths of vectors (so implementation dependent).
SRFI 231-bis could have
(compute-broadcast-interval intervals)
We don't need the broadcasting argument because we have only one value
(#t, we won't have 'permissive) for the broadcasting argument.
math/array also has
(array-broadcast arr ds) → (Array A)
arr : (Array A)
ds : Indexes
SRFI 231-bis could have the similar
(array-broadcast array new-domain)
Array broadcasting can conceptually be implemented by a many-to-one
affine map on multi-indices.
I think array-broadcast should be treated like array-reverse,
array-permute, array-extract, etc., which are simple array transforms
that can be applied to generalized arrays.
*** Adding axes to arrays ***
Racket's math/array has
(array-axis-insert arr k [dk]) → (Array A)
arr : (Array A)
k : Integer
dk : Integer = 1
which returns an array with a new kth axis inserted into arr, by default
of length 1. If dk > 1 is given, then arr is broadcast along the new
axis dk times.
In math/array, when dk is 1, array-axis-insert returns the same result as
(array-list->array (list arr) k)
In SRFI 231, to get (almost) the same result, one could
(array-stack k (list arr))
In SRFI 231, array-stack always copies the argument arr to a specialized
array; in Racket's math/array whether the result is strict or non-strict
depends on the parameter array-strictness.
That's interesting, when passed a single element in a list, array-stack
could just as well create a new indexer instead of copying the array.
And that is exactly what
(specialized-array-reshape arr new-domain)
would do, because reshaping an array by adding length-one axes to the
domain always succeeds.
So perhaps SRFI 231-bis should have a separate routine
(array-axis-insert k arr)
that just re-indexes the elements of arr, whether arr is
immutable/mutable or specialized/generalized.
*** Broadcasting and mutability ***
With *implicit* broadcasting, the broadcast version of the array
arguments are "consumed" by the called procedure immediately after being
broadcast. There is no reason to make the broadcast array mutable.
With *explicit* broadcasting, the broadcast version of the array is
available for later manipulations.
So for explicit broadcasting, the result could inherit the mutability of
the argument. Changing one array element of a broadcast array could
change other array elements, but we'll assume that the programmer
understands this.