Very good points! Thanks for writing this.

> Complaints ahead.

I like complaints, because they usually convey something necessary.

> I think we should first and for most define the mission of the schemedoc
> effort.
>
> The srfi website claims the following:
>
>  > collecting, organizing, and serving indexes of Scheme code found in
> SRFIs, Scheme implementations, RnRS standards, etc.
>
> Is the above mission statement taken from the SRFI website still valid?

This started more as a gathering point or community or like-minded
people than a well-defined project with a limited mission. Indeed, the
topics on the list keep expanding from the above description. I thought
that was OK, but if it's easier to limit the scope I don't mind it.

The way I've viewed it, a particular product (the api.schemers.org
domain, or the docs.schemers.org domain, or a particular Scheme library)
would have a fairly well-defined purpose or mission, whereas social
forums like this mailing list or the GitHub organization are more of a
loosely organized meeting point and don't have a clearly limited scope.
But that was just my impression.

>  From recent discussion, it seems much more than that. Since it includes
> at least getting together a cookbook, I proposed a search engine too.
> we started the awesome-scheme repository.
> I think that is all, I take the blame for opening the pandora box about
> something more akin to a community website even peer-to-peer stuff...

I thought from the start that these were supposed to be such
communities, so I was happy that you proposed the Awesome Scheme list
for example :)

>     Yesterday I also wrote a long reply to Amirouche's mail
>
> No worries, you can still send it, privately if you prefer, please.

I will, there was nothing private in it :) I just got overwhelmed with
the amount of work we have. Email can be very rewarding but it also
takes a lot of time from hacking.

> That's the thing that I would like to avoid. There is a lot of competition
> already in Scheme community between implementations and sadly
> also between people.
>
> If we can agree on something, our proposal will have more strength.

You raise a very good point. Maybe it _is_ better to have only one thing
and work in close cooperation. Scheme has had a lot of separate groups
in the past. That has produced good work, but it hasn't produced unity.

It's also worth working on. For most good programmers, it's easier to
work on the technical aspects than social ones. So if social things like
common goals are difficult to work out, people go by themselves to work
on technical stuff. It would do us well to try to resolve any
differences early, as you say.

And the most successful programming languages like PHP, Perl, Java
succeed largely because their programmers are great at working together.
PHP has probably the most positive and helpful community I've been in
(most languages have a reasonably good community nowadays, but the PHP
people just go out of their way to get things done and help each other).

> I don't want to disjoint efforts. I have done too much of that already.

OK, you are right.

> By the way, I think that running biwascheme in nodejs to expose a
> graphql endpoint is not a good idea: a) biwascheme is basically a toy
> until JS VM implement tail-call-optimisation, which is not happening any
> time soon.

We looked at Biwa but it isn't trivial to set up in its present form.

In any case, if we ran Biwa on Node, it would just be a little glue code
for Node's web and GraphQL libraries. If those were written in Scheme,
then we should probably use a native Scheme implementation like Chez
instead of Node.

> b) graphql will slow down the overall progress because we
> need to understand how it works and same for people that would like
> to consume the API. Since there is no existing library, it is far from easy
> to get started.

For me, it has been a huge joy and relief without which the project
would be dead in its tracks. They simply did a great job solving a very
difficult problem. There is absolutely no way we could make anything as
good from scratch, even if we spent a lot of time.

Basic GraphQL clients are easy to make - the complexity is on the server
side. As a client, you just send a JSON query as a HTTP request, then
read a JSON response back. A curl one-liner can do it. I'll write a
client library for Racket soon. If you have other Schemes with easy HTTP
client and JSON encode/decode, I can port the library to it also.

> c) running nodejs doesn't deserve an explanation why it is
> a bad idea. It will send a bad signal to the users. SchemeDoc should
> promote Scheme, preferably scheme that actively support Scheme.

Starting with Node is not a bad idea and only some trolls will mock us
for it. Most people are realists and understand that we can't do an
infinite amount of work in a few months.

Currently we just have Scheme scripts written using Racket and Gauche
that I modified to generate JSON. The API server is a trivial JavaScript
program that reads the JSON files and serves them using the Node graphql
library (which is extremely easy to use). We're not running any Scheme
on Node right now. We got tired of the constant setbacks for like 2-3
months and decided to boost morale by shipping something that works.

Ultimately we cannot promote Scheme if we can't keep our own workload
reasonable. It's best if we simply admit that Scheme doesn't have
anything as good as GraphQL right now, and work on writing libraries to
implement it. When those libraries are ready we can switch to them. But
doing our own API from scratch and ignoring the industry's lessons just
so we can say everything is in Scheme right away instead of waiting a
few months, seems like a recipe for burn-out to me. I mean, our web
proxy is Nginx and our OS is Linux, those aren't written in Scheme
either (though we do actually configure them in Scheme).

No-one should sacrifice any of their mental and physical health for
technical purity (and it's not the fastest way to solve any technical
problems, like for instance offering compelling HTTP server and GraphQL
support in Scheme, both of which many people would love to use
eventually, in completely different projects than ours).

> Obviously I would prefer to continue to work on Arew Scheme and continue
> to work in Chibi Scheme.

I didn't know you have a Chez distribution, looks great!

I'd like to have more libraries that are portable to many Schemes, and
I'm willing to work on that gradually. Recently Göran has been doing
great work with portability around Akku, and Alex around Snow.

> In another mail I did not send (!) I was arguing that requiring a
> backend software would be difficult to maintain.

The current API is extremely easy to maintain. The server program is
trivial - it just reads some JSON files into a GraphQL schema.

We should definitely serve S-expressions, but we don't have the human
resources to do everything right away. The original plan was to make a
custom S-expression API from the start, but I soon realized we would hit
a Greenspun's Tenth Rule of APIs, where we would keep re-inventing
non-standard versions of GraphQL features poorly. That's why GraphQL was
chosen - _after_ having a bad experience trying to do things manually.
It has competitors like OData but those seemed like more complex and
less elegant versions of the same basic approach.

IMHO the best thing would be to start with standard GraphQL, study it
well, and then make a S-expression syntax and pure-Scheme engine for it.
That syntax and engine could be a re-usable Scheme library so that other
programs can use it too. The Scheme community has several people who are
experienced with and interested in graph databases like you, Arthur and
John, so we should definitely be able to do a good job.

> So, the idea that popped is to use git
> to store the data
> and build a new version of the website when it is merged in master. This
> makes
> contribution from developers easy (they just need git and an editor), it
> is easy to
> maintain, it is fail-safe in the sens that the schemedoc or the server
> disappear
> there will be copies around that will be able to re-launch the project.
>
> It might seem dull to rely on git, but we can be creative in this regard and
> store more than simple markdown files and instead rely on s-expr to
> represent
> more complex data, like scheme metadata repo does.

A static dump of the whole API data is a good idea (even Wikipedia
offers a static dump of all their articles!) But they are not a good
idea for the primary interface because most of the time, only a bit of
data is needed.

The API source data is already S-expressions, and the plan is to keep it
that way. Frankly, I don't want to think about the details too much,
because I'm not capable of doing any more work than I do now. It's not
possible to coordinate or implement anything useful around the API
faster than is already being done. I don't believe it's possible to do
better than we are doing without devoting one's whole life to that one
project, which no-one sane is willing to do :)

> I think we should *define* the mission of the website:
>
> According to (first paragraph) wikipedia page about mission statement,
> we should have a few key points explained out:
>
>   * Key market: the target audience
>   * Contribution: the product or service
>   * Distinction: what makes the product unique or why the audience
>     should buy [or use] it over another
>
> https://en.wikipedia.org/wiki/Mission_statement

Personally, I don't believe in working out mission statements. I've seen
many, and I think they are only useful when they say something that the
people already believe in, as an inevitable result of their experiences
and convictions, without thinking about any mission or strategy at all.
The most passionate mission statements are called manifestos, which may
be a little too much passion for projects of this scope :)

> I have plenty of time and I want to invest it correctly.

I'm glad that you are here :) You are very talented and driven. I like
strong opinions too. I'm not bothered by disagreement.