> Indeed. I use "nohup command >output &" (you need the &) when starting > a long-running job on a remote system, so that if the ssh connection > falls down, "command" will continue to run, and I can consult "output" > when I reconnect, perhaps much later. tmux is probably the default thing to use nowadays. But yes, one-off batch jobs like that would probably be the intended use case of nohup. > I do want syslog, but in a separate SRFI. I'm neutral on that. The syslog protocol is simple. > They should just write all their logs to stdout/stderr like normal > programs, > > Just logging to stdout/stderr is really not adequate. Syslog lets the > application specify an arbitrary number of discrete log streams for > different purposes, which can be shared with other copies of the > application or indeed completely different applications. Furthermore, > the application need not be a daemon: it can be a batch process or an > ordinary user program being debugged. > > In addition, a severity level (emergency, alert, critical error, error, > warning, notification, information, or debugging information) is > associated with each message sent. The stream ID and severity, along > with the text, are sent by a standard wire protocol to a syslog server > (there are a large number of standards-conformant syslog servers). Such > servers can be anything from dumb file writers to multiplexers similar > to daemontools, but with the above information available to assist > selective logging, ignoring, text messaging, or what not. > > It's this sort of thing that makes me and others dislike djb's software: > he thinks he can reinvent the wheel. Sometimes he can; sometimes his > wheel is hexagonal. I disagree with all of that (and disagreement is fine). The reinventions are due to specific problems with the prior wheels. In this case, it was syslog that reinvented the wheel of using stderr for errors; daemontools merely went back to the original wheel. The Unix philosophy is that each program does one small thing well. Writing errors to stderr is a textbook application of this philosophy. Separate programs can be written to read stderr and timestamp/route/filter/store it. The resulting system is simpler, more reliable and more flexible than baking fancy logging separately into every program (even if you are using a logging library). Mandating a special logging system will never compose as well as plain old write(). Heroku has scaled this stderr system up to an entire automated server platform with tons of customers and third-party service providers. It works extremely well. The 12factor page (<https://12factor.net/logs>) provides a clear overview of how this works. They also built a tool called Foreman that simulates Heroku's scaler for local development (it's a mini-daemontools in one process). That program collects all the logs from its subprocesses and dumps them into the user's terminal instead of routing into Heroku's cloud. From this perspective, syslog is the hexagonal wheel. It's great that it has spawned a community of tools, but the abstraction seems inverted: sending logs to a socket is a fundamental building block of writing logs to files or writing them to stderr? In fact, OpenBSD added a sendsyslog() system call (yes, in the kernel) to make syslog more reliable. And syslogd uses a special device and ioctl() to listen to those messages. It doesn't seem like sound design that the architecture requires something as fancy. Re: multiplexing log streams, many apps are moving to JSON log entries for easier parsing (particularly for large-scale log analytics and searches). JSON lines <http://jsonlines.org/> is a simple and effective way to do it - each line of text is a JSON object. The object can have any log level and stream identifiers that are useful. I don't remember which log entry format Heroku uses, but log multiplexing fundamental to the platform. As for log levels, here's a thought-provoking read from Dave Cheney: <https://dave.cheney.net/2015/11/05/lets-talk-about-logging>. He wonders whether there should be any other log levels besides "debug" and "info".
