The archive file SRFI/implementation is coming along nicely but I'm
getting a bit bored with the lack of variety so I just can't resist
proposing the next thing.

Is there any kind of standard for HTTP server middleware procedures?
This would make a great SRFI - nowadays an almost universally useful
abstraction, with a simple interface permitting numerous implementation
strategies.

As you likely know, there are such middleware standards in other
languages and they are extremely successful there:

* Clojure - Ring (https://github.com/ring-clojure/ring)
* Ruby - Rack (https://rack.github.io/)
* Python - WSGI (https://wsgi.readthedocs.io/)

According to Wikipedia there's also:

* PSGI – Perl Web Server Gateway Interface
* SCGI – Simple Common Gateway Interface
* JSGI – JavaScript web server gateway interface

There's a good chance you have already used one or more of these and
know how much they help smooth out some of the rough edges of web app
development and deployment.

I'll just steal the blurb from Clojure: "By abstracting the details of
HTTP into a simple, unified API, Ring allows web applications to be
constructed of modular components that can be shared among a variety of
applications, web servers, and web frameworks."

The abstract API is basically just:

     (define (my-fancy-request-handler http-request-object)
       (make-response-object-from-request-object http-request-object))

Half the point is that the API doesn't say where the HTTP request comes
from and where the response goes. So applications/middleware conforming
to this interface can be used on top of web servers implemented any
number of ways (CGI, FastCGI, SimpleCGI, Apache or Nginx module, TCP
socket made by a pure-Scheme web server, etc.) If I promise to write all
my handlers so they are stateless, I can also give them to a web server
that serves requests in parallel (using threads or subprocesses) without
worrying about concurrency in my application code.

The other half of the point is that handlers compose. So we can put
logging, monitoring, security, etc. middleware in re-usable libraries
and publish them in a package manager. E.g. if I have a web app and want
to log all its traffic in Apache Common Log Format, I could just install
a common-log-middleware library that somebody else defined like this:

     (define (common-log-middleware log-output-port handle)
       (lambda (request)
         (let ((response (handle request)))
           (write-string (common-log-line request response)
                         log-output-port)
           response)))

And then I would replace this in my webapp:

     (run-web-server my-webapp-handler)

With this:

     (run-web-server
      (common-log-middleware my-log-port
                             my-webapp-handler))

It's possible (and often done) to stack middlewares like this:

     (run-web-server
      (logging-middleware
       (error-recovery-middleware
        (access-control-middleware
         (my-app-request-router)))))

There's a small caveat here - these handlers are based on a serial
Unix-pipe-like one thing in -- one thing out model. This is a great fit
for the current HTTP/1.1 protocol but future versions of HTTP will
likely parallelize things much more, so that e.g. a response can start
being written before the request is fully read; or several responses to
one request can start going out in parallel (e.g. all the static assets
for a website would start streaming at once, and a bit later some
results from database queries would go out in their own streams, all
independent of each other). AFAIK the high-level abstractions for
programming that stuff have not stabilized yet.

I would still specify this serial interface similar to what the other
languages have. It is an extremely simple and powerful abstraction, and
ubiquitous for now. It will also probably be just fine for small-time
websites even in the future. We can write another SRFI about a parallel
web interface in about 5 years once the dust has settled.

What do you think?

KR, Lassi