Re: Want to kickstart the Scheme API already? Lassi Kortela 25 Apr 2019 20:38 UTC

> |     ;; R5RS metadata as S-expressions:
> |
> That one is my favorite.  That can be downloaded once (or infrequently)
> and the client can get nearly instantaneous responses from then on.
> Figuring out the details of that metadata format, or something like it,
> seems necessary for handling many of the other HTTP requests, anyway, so
> it seems like a good investment to work on that first.

I agree that it would be good to provide those big files that have
everything. But small HTTP requests can also be quite fast nowadays. The
beauty of these APIs is that we can make everybody happy -- the big
files can be the source material, some URLs can get only part of it.

A good example is the Wikipedia API. You can easily get the wikitext for
an entire article:


Or only a particular section of that article:


We could have the same -- e.g. if we manage to get good conversions of
the RnRS documents to strict HTML-like markup, we can have API endpoints
to get only particular sections from those documents.

The same principle goes for formats -- we can have S-expressions as the
source format, but it does not harm if we also serve JSON to people who
want it. It's just another API endpoint or parameter.

It's probably less necessary to serve smaller slices of the big
documents for performance reasons now. But if we do it anyway, it helps
make the API more conversational. This sounds like a really weird
concern until you think about it for some time and let it sink in. A lot
of problems in design boil down to problems in having conversations. If
you give something an easy name, it becomes easier to talk about it. And
when it becomes easy to talk about an idea, people start doing
experiments with that idea, which then leads to bigger ideas. If you
only serve things in big opaque chunks, those chunks don't become
conversation pieces and people don't start puttering around with them. A
lot of the success of Unix pipes and REST APIs could be pinned down to
their fundamentally conversational nature. It's so natural to write
pipelines or URLs to access things, that as soon as you get an idea it's
just as easy to write it down in precise technical notation. And then
it's executable and other people start using it and modifying it. It
starts a conversation -- first in your own head and screen, then with
others. This has profound implications for library and language design
too. I believe some Unix people use the word "discoverability" to mean
something similar to what I mean with conversationality. Anyway, just
had to get this design philosophy interlude out of my system :) Back to
regularly scheduled programming.

> Of course, the metadata format should include some sort of version
> information for the format, as well as a timestamp indicating when the
> metadata was captured.

These are very good ideas.