> From: Jim Blandy <xxxxxx@redhat.com>

    > Jim Blandy <xxxxxx@redhat.com> writes:
    >> Well, if SRFI-50 turned out not to be what I was hoping, and I didn't
    >> come to my senses quickly enough, I was going to turn <minor/minor.h>,
    >> into a .texi file, start a SRFI from that, and see what people said.

(As I said, I'm willing to work on a SRFI too -- and have Pika docs I
can contribute for that.)

    > In light of that, I'm curious to know how people generally feel
    > about the Pika vs. JNI issue.  If there's a near consensus on
    > one or the other, then that could save a lot of trouble.

    > Here's how I see it:
    [....]
    > Is this fair?  What have I missed?  What do people think?

I think it's pretty good though I do have some replies.  I'm replying
to your message a bit out of order.

    > It would be nice to see sample code in each style.  C implementations
    > of "cadr" and "assq" would be nice.  As far as I know, error checking
    > is similar under both interfaces, so that can be left out.

    > [cadr isn't very interesting, imho -- cadr example snipped]

    >     mn_ref *
    >     assq (mn_call *c, mn_ref *key, mn_ref *alist)
    >     {
    >       while (mn_pair_p (c, alist))
    >         {
    >           mn_ref *pair = mn_car (c, alist);
    >           mn_ref *pair_key = mn_car (c, pair);
    >
    >           if (mn_ref_eq (c, key, pair_key))
    >             return pair;
    >
    >           mn_free_local_ref (c, pair);
    >           mn_free_local_ref (c, pair_key);
    >           alist = mn_to_cdr (c, alist);
    >         }
    >
    >       return mn_false (c);
    >     }

Note, by the way that while my assq is arguably uglier than yours,
mine is O(1) space and yours is O(N) where N is the length of the
alist.  (Does that count as a benefit of having more hair on my assq?
(sorry :-)).

/* untested code */
t_scm_error
scm_assq (t_scm_word * answer,
          t_scm_arena instance,
          t_scm_word * key,
          t_scm_word * alist)
{
  struct assq_frame
  {
    SCM_FRAME;
    scm_word_t * pair;
    scm_word_t * key;
  } l;

  SCM_PROTECT_FRAME (l);

  scm_make_false (answer, instance);

  while (scm_is_pair (instance, alist))
    {
      scm_car (&l.pair, instance, alist);
      scm_car (&l.key, instance, &l.pair);
      if (scm_values_eq (instance, &l.key, key))
        {
          SCM_LSET (answer, instance, &l.pair);
          break;
        }
      scm_cdr (alist, instance, alist);
    }

  SCM_UNPROTECT_FRAME (l);
  return 0;
}

(I should be making a new release of Pika soon which contains lots of
example code.)

    > Commonalities:
    > - Both work by having C code manipulate only references to Scheme
    >   values, not Scheme values themselves.
    > - Both impose few restrictions on the representation of Scheme objects.
    > - Both allow GC to occur at any time.
    > - Both can be implemented in a way that interacts nicely with threads.
    > In Pika:
    > - Leaks are impossible, since references are stack-allocated.
    > - References are freed upon exit from the lexical block that owns
    >   them --- finer-grained than JNI-style.
    > - Probably less overhead than JNI-style.

All that seems right to me.

    > But:
    > - Forgetting an UNGCPRO corrupts the GC's data structures, and may
    >   fail only intermittently.  Irregular exits (return; goto; break;
    >   continue) require attention.  [...]

A lint-like program can address that issue if it proves serious.

    > 				[...] longjmp is even harder.

It's not that hard -- you just can't use "naked" longjmp to jump past
protected frames (you would have to use a `scm_setjmp' / `scm_longjmp'
pair.)  So, how much of a restriction is that?   Here's what I see:

Let's consider three classes of C function:

   SCHEME	-- functions provided by the FFI implementor
   FFI-USER	-- functions written to use the FFI
   3RD-PARTY	-- other functions, such as those in a C library
                   to which we are making bindings using the FFI

SCHEME and FFI-USER functions can simply be written to use
`scm_setjmp/scm_longjmp' rather than `setjmp/longjmp'.   There's no
problem among those.

I think it would unduly burden FFI implementors to permit naked
longjmps past SCHEME frames.   Is that a controversial belief?
In other words, suppose that we have a call-chain like:

	3RD-PARTY	-- calls setjmp
        SCHEME
        3RD-PARTY	-- calls longjmp?

In that kind of call chain I think that the longjmp should be declared
illegal by the FFI spec.

Suppose we have a call chain like this:

	FFI-USER	-- wants to fill a jmp buffer
        FFI-USER*
        3RD-PARTY	-- wants to lonjmp to the outermost

That can be accomplished by wrapping the 3RD-PARTY code:

        FFI-USER	-- uses scm_setjmp
        FFI-USER*
        WRAPPER		-- uses setjmp and scm_longjmp
        3RD-PARTY	-- uses longjmp

so that a longjmp from 3RD-PARTY to the outermost FFI-USER is
accomplished in two steps: a longjmp to WRAPPER which immediately
calls scm_longjmp.

The wrapping can work the other way too:

	3RD-PARTY	-- uses setjmp
        FFI-USER*
        FFI-USER	-- wants to use longjmp

becomes:

	3RD-PARTY	-- uses setjmp
        WRAPPER		-- uses scm_setjmp and longjmp
        FFI-USER*
        FFI-USER	-- uses scm_longjmp

The case remaining is where the FFI-USER code has no access to the
jump buffer:

	3RD-PARTY	-- uses setjmp
	FFI-USER*
	3RD-PARTY	-- uses longjmp

but that can be handled by a wrapper too:

	WRAPPER		-- creates an auxiliary stack
        3RD-PARTY	-- uses setjmp
        FFI-USER*	-- stores frames that might be lost in
                           a longjmp on the auxiliary stack
        3RD-PARTY	-- uses longjmp

which is very similar to what I would expect a JNI/Minor-style
implementation to do but only pays for the costs of the auxiliary
stack when it is actually needed.  This approach is no different from
the kind of thing you'd need to do if, for example, one of the
intermediate FFI-USER* frames wanted to open a temp file that it might
not otherwise get a chance to close.

I would not object to a Pika-style FFI which additionally contains a
standard interface to such "auxiliary stacks" -- so that authors of
FFI-USER code could make longjmp-safe functions that can be freely
combined.

Of course this case:

	FFI-USER	-- uses scm_setjmp
        3RD-PARTY
        FFI-USER	-- uses scm_longjmp

depends on the nature of 3RD-PARTY but if 3RD-PARTY can handle being
longjmped past, then at most you would add wrappers around it.

    > - Since the API functions all expect pointers to t_scm_word values,
    >   this discourages people from passing them around directly, but it
    >   can still be done --- e.g. "frame.x = frame.y;" --- and doing so
    >   will usually work.  But doing so is a bug.

I believe that we can provide a header file which will declare the FFI
types and functions in such a way that correct FFI-using code will
compile with this header (but produce nonsense code) and FFI-using
code having the kind of bug you mention will cause a compiler error.
In other words I think that we can use existing C compilers as a kind
of `lint' for that particular problem.  (It won't catch a use of
something like `memcpy' rather than C assignment, of course.)

Also, that kind of bug is again something that could be caught fairly
easily by a custom lint-like program.

    > - Variable declarations are cluttered with enclosing structs and GCPRO
    >   / UNGCPRO calls.

Tastes vary.   In isolation, at least, that seems a weak reason to use
a more costly approach.

    > - Functions may only return Scheme values by reference; they may not
    >   provide them as their (syntactic) return values.  Instead of writing
    >   "f (g (x))", you must write:

    >     g (&frame.x, &frame.temp);
    >     f (&frame.temp, &frame.temp2);

    >   In other words, you must write your code as linear series of
    >   operations which work by side-effects.

    > In JNI-style:
    > - Functions can return references directly, so code need not be
    >   linearized.  You can write "f (call, g (call, x))" --- if you know
    >   that "call" will return and free g's return value soon enough.
    > - Local references are freed automatically when the Scheme->C call to
    >   which they belong returns.  Leaks due to unfreed local references
    >   (which will probably be the most common sort of error) have a
    >   bounded and often (though not always) short lifetime.

One thing that bothers me about the JNI-style:

	f (g(x), h(y))

is the lifetime of the references returned from `g' and `h'.  If that
call is in a loop, for example, two new references will be allocated
on every iteration and, absent the programmer taking steps to
explicitly manage them (sacrificing the gain of the function call
syntax) all of those references remain live until C returns back to
Scheme.

As a side effect of that, functions which should consume O(1) space
will, in basic JNI-style, consume O(N) where N is the number of
intermediate results they create during computation.

Now I concede that for something like `cadr', such space performance
is basically inconsequential (unless `cadr' is itself called in a
loop).  But if I'm writing something more involved I'll immediately be
back to either the "linear style" of Pika or to something equally
"ugly".

This also comes back to error handling.  We haven't talked about it
much yet but my thinking is that nearly every FFI function should be
able to return an error code and nearly every call to an FFI function
should be checked.  For example, if `scm_car' is passed a non-pair
argument, I think it should be able to return an error code.  Normally
I'd expect people to write not:

	g (&frame.answer, instance, &frame.x);
        f (&frame.answer, instance, &frame.answer);

but:

	err = g (&frame.answer, instance, &frame.x);
        if (err)
          {
            ....;
          }
        err = f (&frame.answer, instance, &frame.answer);
        if (err)
          {
            ...;
          }

or

        /* We know that no errors can arise.
         */
	(void)g (&frame.answer, instance, &frame.x);
        (void)f (&frame.answer, instance, &frame.answer);

or something in between.

Now, to be sure -- we can not _require_ implementations to notice "it
is an error" conditions (such as passing an integer to CAR) -- but
since implementations are encouraged to, and since many or most
implementations do so, standardizing the C interface to such errors
seems a reasonable thing to do in an FFI.

    > - No GC data structures live on the C stack, so careless control flow
    >   and longjmps will not corrupt the GC's data structures.
    > - The "explicit free" model is familiar to C programmers.

But, again -- it's the lifetimes of intermediate results that I find
problematic (along with the costs of allocating them).  The "explicit
free" model is indeed familiar but that's not quite what you're
proposing.

    > - Variables are declared normally, and their values used directly.

Variables are declared normally in Pika, too.  I think you mean that
JNI-style attempts to disguise handles as Scheme values.  It is
because it can't pull off that illusion perfectly that I think it is a
questionable choice.

    > - Since mn_ref is an incomplete type, it can't be dereferenced, so
    >   people can't be sloppy and operate on the heap values directly.

And, again: (a) I'm fairly certain we can trick existing C compilers
into acting as a lint for that particular kind of error;  (b) an
extensible lint that can be programmed to check for those and other
kinds of errors is a tractable problem and as much of a "killer app"
as you're likely to find in the development-tools world.

    > But:
    > - The "explicit free" model is still error-prone.  The fact that leaks
    >   are bounded by their owning call's lifetime may not always
    >   help.

Indeed.

    > - Probably more overhead than Pika-style.
    > - Code will be cluttered with explicit-free crap.

-t