My current proposals for a binary and equivalent text format to carry dynamically typed data are here:

https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/CoreAsn1.md

https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/CoreSexps.md

http://tinyurl.com/asn1-ler

A new name is needed for this: I've been using "core" but the extended version (al types) is now a simple extension of the core (essesntial must-support types), as shown by the Core column in asn1-ler.  I'm currently using "Lisp Serialization Conventions", but it's clunky.


On Sun, Oct 6, 2019 at 9:46 AM Amirouche Boubekki <xxxxxx@gmail.com> wrote:
Le dim. 6 oct. 2019 à 14:41, Lassi Kortela <xxxxxx@lassi.io> a écrit :
>
> Your peer-to-peer proposal is awesome, but also quite ambitious.

Yes, it is not a simple or easy matter. But it is also lighter than
the competition.

> Is there really one true way to do peer-to-peer?

As usual, I had a look around. TL;DR: There is no other way, so far.

- IPFS: Only the JavaScript (and maybe the Go) implementation are in a
working state.
  The real problem with IPFS is that the specification / documentation
is not good enough.
  Last time I checked, around 6 months ago, I could not implement IPFS
just based on those.
  Also see https://github.com/orbitdb/orbit-db

- gnunet: I was heavily invested prior to 0.11.0 release. gnunet is
the only project that is both
  portable across operating systems and that one can bind using C
Foreign Function Interface.
  The main problem with gnunet is that it is _very slow_. Also, it
took me less time to create the
  prototype mentioned previously. Otherwise said, it is difficult to
work with gnunet.

- Freenet is based on the JVM.

- GNU Jami is coded in C++ and last time I checked it doesn't have C wrappers.

- DAT and Scuttlebutt are not documented and running on top of NodeJS.

- WebRTC requires a server to work.

Most of peer-to-peer work and knowledge was invested in libtorrent
(C++ with abandonned C wrappers). There is a specification, but so far,
libtorrent doesn't support BEP-50 (pubsub) and BEP-46 (mutable dht).

[0] http://bittorrent.org/beps/bep_0050.html
[1] http://bittorrent.org/beps/bep_0046.html

Also https://en.wikipedia.org/wiki/Bencode.

As far as I understand, to be able to offer the features required to
build (rich internet)
applications that are more complex than file sharing, one needs to
have control over
the code of the peer. And that is without mentioning some kind of
security or network
health.

I am not trying to convince anyone that prior art is full garbage.

So far, peer-to-peer tools were not deployed in a way that would allow
one to say
that there is an industry standard. Torrent protocol is too restricted
to build the kind
of application I am thinking about. The common pratice is to rely on kademlia.

They are things to learn from existing projects, but as far as I
understand, there is also
room for innovation(s).

> I don't have any expertise on the topic, so I'll defer to others.
>
> > I meant to write that many other SRFIs should or could be written
> > before this one. Here is the one I can think of:
> >
> > - UDP or TCP or Web sockets
>
> Prior art:
>
> - SRFI 106: Basic socket interface (2013)
>
> - Quite complete BSD sockets API for Chicken:
> https://wiki.call-cc.org/eggref/4/socket I've been thinking of writing a
> full BSD sockets SRFI based closely on this. The Windows WinSock API is
> a close copy of BSD sockets so it would have to work identically on that
> as far as possible.
>
> - Chez Sockets: <https://github.com/arcfide/chez-sockets>. IMHO this API
> is very hard to use.
>
> - Gambit has sockets as device ports:
> <http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Network-devices>
> and DNS lookup starting at
> <http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Service-information>.
>
> - Gauche BSD sockets wrapper:
> <https://practical-scheme.net/gauche/man/gauche-refe/Networking.html#Low_002dlevel-socket-interface>
>
> - Taylor Campbell's comments on lackluster socket APIs. Search for
> "2006-09-16 On networking interfaces in Scheme" at
> <https://mumble.net/~campbell/blag.txt>.
>
> To Taylor's comments, I would add that the address/port/socket-options
> interface is hard to do well. It tends to be either too simple or too
> complex. I have an idea but haven't had time to test it.

Indeed. I like that paragraph:

>  Unfortunately, programmer laziness in conjunction with a misguided
>  desire for `nicer' interfaces tends toward not supporting any more
>  esoteric capabilities of BSD sockets or flexibility for later
>  extension or protocol independence.

I made a mistake in the early draft, send and recv procedures are not enough.

> > - Binary scheme expressions
>
> If you mean writing S-expression-equivalent data in a binary format, see
> long conversations between John, Alaric and I on schemepersist mailing
> list. Working code at <https://github.com/lispunion/database-subprocess>
> in files <binary.*>. We're debating switching to an ASN.1-based format.

Thanks I will look around.