> On Nov 8, 2022, at 2:27 AM, Marc Nieper-Wißkirchen <xxxxxx@nieper-wisskirchen.de> wrote:
>
> Am Mo., 7. Nov. 2022 um 21:08 Uhr schrieb Marc Feeley <xxxxxx@iro.umontreal.ca>:
>>
>>
>>> On Nov 7, 2022, at 1:43 PM, Marc Nieper-Wißkirchen <xxxxxx@nieper-wisskirchen.de> wrote:
>>>
>>> Thank you!
>>>
>>> Am Mo., 7. Nov. 2022 um 19:19 Uhr schrieb Marc Feeley <xxxxxx@iro.umontreal.ca>:
>>>>
>>>>
>>>>> On Nov 7, 2022, at 4:41 AM, Marc Nieper-Wißkirchen <xxxxxx@nieper-wisskirchen.de> wrote:
>>>>>
>>>>> Hey Marc,
>>>>>
>>>>> could you describe the exact semantics of Gambit's thread-interrupt!
>>>>> or give me a link to where it is documented?
>>>>>
>>>>> Specifically, what happens when the thunk returns normally?
>>>>>
>>>>> We talked about capturing a continuation inside the thunk, which is
>>>>> related to the question.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Marc
>>>>>
>>>>
>>>> The API and semantics of thread-interrupt! has evolved over time since the introduction of threads in Gambit v4.0 (~2000).  The basic idea is to force a runnable or blocked thread to immediately execute a call to a thunk, regardless of what that thread is currently doing.  Conceptually each thread executes a series of atomic actions (“atomic” in the sense that they are operations at a certain level of abstraction of the virtual machine, such as a call to “cons”, “car”, “pair?”, etc).  But note that some Scheme predefined procedures, such as “append”, “map”, etc are not atomic and are a series of atomic actions.  The thread interrupt mechanism inserts the call to the thunk between such atomic actions thus ensuring that interrupts happen at “safe places” (for the Scheme virtual machine, which does not mean that it is safe for the logic of the program, which is the programmer’s concern).  Conceptually, if the thread was about to evaluate <expr>, it replaces this by the evaluation of (begin (thunk) <expr>) so that the thunk’s result is ignored and the thunk is called with the thread’s current continuation as a parent, including the dynamic environment.
>
> The problem I still see is that thread-interrupt! and, to a lesser
> extent, thread-exit! will expose implementation details of standard
> procedures (those that are implemented as non-atomic ones).
>
> Assume that we have a procedure "foo" in the standard, that, by
> definition, increases the global variable "x" by one and then the
> global variable "y" before it returns.
>
> With the usual "as if" rule, it wouldn't matter if foo were implemented as
>
> (define foo
>  (lambda ()
>    (set! x (+ x 1))
>    (set! y (+ y 1))))
>
> or as
>
> (define bar
>  (lambda ()
>    (set! y (+ y 1))
>    (set! x (+ x 1))))
>
> With thread-interrupt!, however, a continuation in the middle of the
> execution of foo could be caught, making the order of assignments
> observable.  This would suddenly rule out the second implementation
> because the specification of "foo" would have become an
> over-specification with the introduction of thread-interrupt!.
>
> Even more complicated is the matter with the non-atomic increase of x and y.

But this issue (exposing the implementation) is already the case currently:

1) when the operation throws an exception and the exception handler captures the continuation (for example imagine the variable x in (set! x (+ x 1)) contains a non-number and “+” raises a type exception, or that x is a bignum and “+” needs to allocate a bignum and this needs to call the garbage collector because the heap is full, and the GC “hook” (a common very useful extension) captures the continuation to show the user where the GC was triggered or for profiling).

2) when the operation includes a procedure call such as (set! x (f x)) where f might contain code that captures the continuation or raises an exception like for #1

3) when “+” has been set! to a procedure that captures the continuation (this may not be allowed by R7RS, but it is in R5RS and in Scheme systems that allow this for debugging reasons)

Moreover it is possible using threads to observe y being mutated before x, so the transformation you mention above is limited to non-threaded code so clearly not in situations where thread-interrupt! would be used.

If it is critical that the two assignments can’t be interrupted in the middle then this should be explicitly enforced with a critical section.  This could be achieved using a boolean parameter object that indicates if interrupts should be handled immediately or deferred until later:

    (define bar
      (lambda ()
        (parameterize ((defer-interrupts? #t))
          (set! x (+ x 1))
          (set! y (+ y 1)))))

When defer-interrupts? is #t any incoming interrupts are put in a queue.  When defer-interrupts? goes from #t to #f the interrupts on the queue are serviced.

It could also be a special form (with-deferred-interrupts thunk) to hide the parameter object, or equivalent mechanism.

>
> Another problem is that the application, including the standard
> libraries (but not the VM) may be in an unsafe state at the time of
> the interrupt.  So any call to a standard procedure may crash the
> system.  But the standard should allow an implementation to offer a
> safe mode.

This is why I say at that point it is the programmer’s responsibility to use this powerful construct correctly (similarly to using first class continuations and assignments together).

>
> Hypothetical procedures "thread-raise" and "thread-raise-continuable",
> would make it a bit easier.  We would then have:
>
> (define thread-interrupt!
>  (lambda (thread thunk)
>    (thread-raise-continuable thread (make-interrupt-condition (lambda
> (exc) (thunk)))))
>
> (define-condition-type &interrupt-condition &condition
>  make-interrupt-condition interrupt-condition?
>  (handler interrupt-condition-handler))
>
> The initial exception handler would then be:
>
> (lambda (exc)
>  (cond
>    [(interrupt-condition? exc)
>     ((interrupt-condition-handler) exc)]
>    [else ...]))
>
> Procedures like foo could then install a custom exception handler
> ("interrupt handler").
>
> This would make thread-interrupt! less powerful, though.
>
> What is a good way out?

Isn’t it still possible to write an exception handler (called at the moment of an interrupt) that contains a continuation capture?  So I’m not sure what is gained by doing it this way.

On a separate subject I have some thoughts about thread-exit!.  I think this procedure should drop the “!” for consistency with the R7RS “exit” procedure.  The semantics should also be modeled after “exit” and “emergency-exit”:

   (thread-exit [obj])
   (thread-emergency-exit [obj])

in such a way that in the primordial thread we have:

   (exit [obj]) = (thread-exit [obj])
   (emergency-exit [obj]) = (thread-emergency-exit [obj])

The thread-emergency-exit procedure would “Terminate the thread without running any outstanding dynamic-wind after procedures” (same wording as “emergency-exit” but “program” is replaced by “thread”).  The optional “obj” parameter would be the thread’s result, accessible with (thread-join! <thread>).

A thread-terminate! procedure would not be needed because a thread could terminate a target thread with either:

   (thread-interrupt! <target-thread> thread-exit)
   (thread-interrupt! <target-thread> thread-emergency-exit)

and the first form would be preferred to let the thread do any required cleanup.  It would also be possible to do (thread-interrupt! <target-thread> thread-exit), then (thread-join! <target-thread> <timeout>), followed by (thread-interrupt! <target-thread> thread-emergency-exit) if the timeout was reached, in case the target thread is wedged.

Marc

>
> [...]
>
>>>
>>> Does Gambit handle signals through thread-int!, e.g. as in the following?
>>>
>>> (##thread-int! <thread> (lambda () (raise-continuable <signal-condition>)))
>>>
>>> or
>>>
>>> (##thread-int! <thread> (lambda () (signal-interrupt <info>)))
>>>
>>> where
>>>
>>> (define (signal-interrupt info)
>>> ((current-signal-interrupt-handler) info))
>>
>> It depends what you mean by “a signal”…  If you mean something related to POSIX signals then no it is not implemented that way because POSIX signals are executed asynchronously (with respect to the Gambit virtual machine) so they could happen in the middle of some VM operation that should not be interrupted because the VM state is temporarily inconsistent.  So instead a POSIX signal will register the signal in a bit set, and then raise an “interrupt flag” that is checked regularly at “safe points” and the handler checks the bit set.  This is a very low overhead polling mechanism that piggybacks on the stack overflow detection logic.  See my paper:
>
> I meant a "VM signal", which could be triggered by a POSIX signal. I
> was interested in how the high-level interface/high-level semantics
> worked.  It seemed to me that thread-interrupt! is a sufficiently
> general primitive.
>
>>   Polling Efficiently on Stock Hardware, FPCA93 (http://www.iro.umontreal.ca/~feeley/papers/FeeleyFPCA93.pdf)
>
> I have been experimenting a bit with virtual machines.  There, I
> usually use hardware detection for a stack overflow.  The overflow
> handler expands the stack but then sets an interrupt flag.  The actual
> code then just has to poll the interrupt flag (some volatile
> sig_atomic_t).
>
>> Fun fact: that paper has achieved a certain notoriety because it is one of the few references in the book “The Java Language Specification” in the section 11.3.2 “Handling Asynchronous Exceptions”.
>
> Cool!
>
> Marc
>
> [...]