Twinjo implementations, subprocess servers, type signatures Lassi Kortela (16 Apr 2021 09:28 UTC)
Re: Twinjo implementations, subprocess servers, type signatures Arthur A. Gleckler (16 Apr 2021 16:50 UTC)
Re: Twinjo implementations, subprocess servers, type signatures Arthur A. Gleckler (16 Apr 2021 17:13 UTC)

Twinjo implementations, subprocess servers, type signatures Lassi Kortela 16 Apr 2021 09:27 UTC

I've had very little available energy to put into coding lately but
things are slowly picking up. Could work on Twinjo (the data model with
equivalent text and binary formats) in the coming days. In particular,
it should have a C implementation, which I can write.

That C library can then be used to write subprocess servers that speak
Twinjo. I'll convert the SQLite server I wrote quite a while back using
varint-based binary S-expressions.

Twinjo subprocess servers could be written for many tasks as an
alternative to gnarly and implementation-specific FFI directly into C
libraries. I think we should specify most Scheme APIs such that both an
FFI implementation and a subprocess implementation is permitted, that
being an implementation detail.

Subprocess API calls should probably be list in, list out. This means
that most Scheme procedure calls such as (database-open "foo.db") can
have a straightforward translation to a Twinjo list which is passed as
binary over the pipe. The server should return a list representing
multiple values such that each list element is one value. This also has
an obvious direct mapping to Scheme where desired.

I'd like to write Twinjo support for ML as well (starting with SML or
OCaml depending on inspiration). For language compatibility beyond
Scheme, each pipe server should have a type signature for its API.

It may be that instead of positional arguments, we should use keyword
arguments in Twinjo calls from the start. Keyword args can be encoded as
a property list alternating between key and value; we could use Twinjo
symbols for the keys. For ML-style type signatures, we can pretend a
remote procedure's arguments are a record, and optional arguments are
presented using an option type (i.e. Maybe).

There should be a standard way to return an error.

There should probably be an optional facility for running more than one
procedure call in parallel, keeping track of which return value
corresponds to which call.

We don't have to stop at OS subprocesses either. Twinjo can be spoken
over the network, so a Twinjo-speaking program can be contacted over
TCP, SSH, or perhaps even HTTP if we can figure out a suitable wrapper
for the latter. If pipe servers simply use stdin/stdout to talk Twinjo,
they can be easily moved into a networked environment by plugging them
into something like tcpserver, inetd, or using a ssh command line to run
them remotely. Hence our pipe servers don't need to do anything special
to care about networking.

Particular pipe servers to practice with, could be written in C:

- SQLite and Postgres
- Compression/archiving stuff: libarchive, lzma, zstd, lz4, ...

Question for John, is your Twinjo spec in a state where implementation
can begin?

The spec doesn't mention Unisig (https://github.com/unisig/unisig) but
we could put one at the beginning of the stream for identification.