> Indeed.  I use "nohup command >output &" (you need the &) when starting
> a long-running job on a remote system, so that if the ssh connection
> falls down, "command" will continue to run, and I can consult "output"
> when I reconnect, perhaps much later.

tmux is probably the default thing to use nowadays. But yes, one-off
batch jobs like that would probably be the intended use case of nohup.

> I do want syslog, but in a separate SRFI.

I'm neutral on that. The syslog protocol is simple.

>     They should just write all their logs to stdout/stderr like normal
>     programs,
>
> Just logging to stdout/stderr is really not adequate. Syslog lets the
> application specify an arbitrary number of discrete log streams for
> different purposes, which can be shared with other copies of the
> application or indeed completely different applications.  Furthermore,
> the application need not be a daemon: it can be a batch process or an
> ordinary user program being debugged.
>
> In addition, a severity level (emergency, alert, critical error, error,
> warning, notification, information, or debugging information) is
> associated with each message sent.  The stream ID and severity, along
> with the text, are sent by a standard wire protocol to a syslog server
> (there are a large number of standards-conformant syslog servers).  Such
> servers can be anything from dumb file writers to multiplexers similar
> to daemontools, but with the above information available to assist
> selective logging, ignoring, text messaging, or what not.
>
> It's this sort of thing that makes me and others dislike djb's software:
> he thinks he can reinvent the wheel.  Sometimes he can; sometimes his
> wheel is hexagonal.

I disagree with all of that (and disagreement is fine).

The reinventions are due to specific problems with the prior wheels. In
this case, it was syslog that reinvented the wheel of using stderr for
errors; daemontools merely went back to the original wheel.

The Unix philosophy is that each program does one small thing well.
Writing errors to stderr is a textbook application of this philosophy.
Separate programs can be written to read stderr and
timestamp/route/filter/store it. The resulting system is simpler, more
reliable and more flexible than baking fancy logging separately into
every program (even if you are using a logging library). Mandating a
special logging system will never compose as well as plain old write().

Heroku has scaled this stderr system up to an entire automated server
platform with tons of customers and third-party service providers. It
works extremely well. The 12factor page (<https://12factor.net/logs>)
provides a clear overview of how this works. They also built a tool
called Foreman that simulates Heroku's scaler for local development
(it's a mini-daemontools in one process). That program collects all the
logs from its subprocesses and dumps them into the user's terminal
instead of routing into Heroku's cloud.

 From this perspective, syslog is the hexagonal wheel. It's great that
it has spawned a community of tools, but the abstraction seems inverted:
sending logs to a socket is a fundamental building block of writing logs
to files or writing them to stderr?

In fact, OpenBSD added a sendsyslog() system call (yes, in the kernel)
to make syslog more reliable. And syslogd uses a special device and
ioctl() to listen to those messages. It doesn't seem like sound design
that the architecture requires something as fancy.

Re: multiplexing log streams, many apps are moving to JSON log entries
for easier parsing (particularly for large-scale log analytics and
searches). JSON lines <http://jsonlines.org/> is a simple and effective
way to do it - each line of text is a JSON object. The object can have
any log level and stream identifiers that are useful. I don't remember
which log entry format Heroku uses, but log multiplexing fundamental to
the platform.

As for log levels, here's a thought-provoking read from Dave Cheney:
<https://dave.cheney.net/2015/11/05/lets-talk-about-logging>. He wonders
whether there should be any other log levels besides "debug" and "info".