Simplified, Limited, Easy FFI: Useful? bear 24 Dec 2003 18:08 UTC


Let's consider the possibilities of a radically different approach.

This is admittedly quite limited, but I think it represents a
"low-hanging fruit" of useful FFI that offers a quite easy
implementation to all POSIX schemes.

I limit my discussion to POSIX environments here, and limit the
proposal to handling scheme calls into other languages, just because
POSIX  provides an easy way to do that and scheme calls into C
code are what I find most useful about FFI's.

What if there was a program running, in a completely separate
process and memory space from the scheme program, that listened
for connections on a port, responded to connections with the
addresses of input and output pipes, took input for a given
call on the input pipe, and wrote results back on the output
pipe?

You could have a scheme function that starts the named program if
it isn't already running, gets the input pipe and writes its args
directly to that port (in some C-friendly data format), and
returns an input port corresponding to the c program's output
pipe, where you can read the result.

This would nail portability, behave correctly with threads, get
the C and Scheme calling disciplines out of each other's hair,
wouldn't screw with garbage collection or continuations, and would
provide an easy point of attachment for any and all C routines.
Additionally, accomodating it would be a small and bounded
problem for implementors, not requiring major redesign of their
runtime.

It would also be straightforward to use the same mechanism to
implement FFI calls to fortran or FORTH or CommonLisp or whatever
other languages, as long as those languages understand ports.

Scheme routines implementing an external call would look
something like....

;; assumes that the program 'cport' can be found on the
;; current path and knows how to do 'foobar arg1 arg2.
(define foobar (lambda (arg1 arg2)
   (with-input-port (FFI_write "cport" 'foobar arg1 arg2)
	(read))))

;; assumes that the program 'forthport' can be found on
;; the current path and knows how to do 'foobaz arg1 arg2.
(define foobaz (lambda (arg1 arg2)
   (with-input-port (FFI_write "forthport" 'foobaz arg1 arg2)
	(read))))

I'll admit freely that it wouldn't provide the flexibility of a
"native" FFI, and wouldn't provide callbacks.  Also, there's a
design choice to be made about whether the 'port' programs can
time out and just terminate themselves, or whether you want them
around to hold persistent state.   But it would be completely
easy to use and so completely easy to understand.  Anybody
could (and would) use it.

I'll also admit freely that if you're going to other languages
for speed this is not likely to satisfy you; the synchronization
and communication will introduce delays.  But usually I find that
I am going to other languages for operating-system calls rather
than faster execution.

				Bear