> I am looking for a supervisor too. At my previous work we used supervisord.
> What do you recommend?

I've had a very good experience with daemontools
(<http://thedjbway.b0llix.net/daemontools.html>). The idea is that it
doesn't use PID files - instead, you provide a run script that starts a
service and leaves it running in the foreground instead of putting it in
the background. There are no PID files -- instead, an always-running
supervisor process tracks the services. Services write log output to
stdout/stderr (instead of syslog or custom logfiles), and daemontools
writes those into logfiles, taking care of timestamping and rotation.

This basic idea has now been re-implemented in runit, s6 and several
other packages which may be more user-friendly and full-featured than
the original daemontools. From skimming the systemd docs, it ought to
even be possible to set up systemd services similar to daemontools run
scripts (i.e. leave daemon in foreground and log to stdout). Systemd has
the advantage that it uses Linux cgroups to kill/signal processes, which
is somewhat more reliable than the portable daemontools way of using
Unix kill(). (In both systemd and daemontools the services are child
processes of the always-running supervisor process.)

daemontools cannot be used as init (PID 1) but runit can. The Void Linux
distribution uses runit instead of systemd as PID 1.

Monit is another supervisor that many people like, but I haven't tried
that one either. I think it comes with its own web dashboard.

I have no experience with supervisord. Did you like it?

>     May I ask if there was a particular reason you switched away from the
>     Guix package manager on your server?
>
> I did not really switch. I plan to use it again. When you have dependencies
> that are not shipped by a regular distro (like wiredtiger 3.2) it is
> very handy to
> have guix to compile it for you. It make updating easier. And rollback much
> easier too.

I can believe that. I've used GNU Stow to manage custom-built software.
Stow is a very simple and effective symlink farm manager but it's always
nicer to have a real package manager.

> When I was using guile, I had several guile package that (obviously) were
> not packaged for ubuntu. So, it made sense to use guix as a package manager.
>
> For my current work, I have a monorepo and all my dependencies are chez
> scheme code. Except wiredtiger. So, I will prolly re-use guix.
If the GUIX package manager is reliable, we could use it on top of a
traditional Linux distro on the api.schemers.org server.

We could also use NixOS (the official Linux distribution of the Nix
package manager) but it doesn't seem to be significantly more reliable
or feature-complete than Guix is. If only we could start the project a
couple of years later, the purely functional distros would be much more
mature by then :) We are at a point in the adoption curve where purely
functional package management seems obviously the future yet the tools
are still lacking some polish.