Email list hosting service & mailing list manager

Size of files.scheme.org Lassi Kortela (06 Jul 2022 11:10 UTC)
Re: Size of files.scheme.org John Cowan (06 Jul 2022 13:07 UTC)
Hosting platforms Lassi Kortela (06 Jul 2022 13:32 UTC)
Re: Size of files.scheme.org elf (06 Jul 2022 22:51 UTC)
Re: Size of files.scheme.org Lassi Kortela (09 Jul 2022 15:49 UTC)
Re: Size of files.scheme.org Lassi Kortela (09 Jul 2022 15:57 UTC)
Re: Size of files.scheme.org John Cowan (10 Jul 2022 02:09 UTC)
Re: Size of files.scheme.org elf (06 Jul 2022 22:54 UTC)

Size of files.scheme.org Lassi Kortela 06 Jul 2022 11:10 UTC

files.scheme.org should one day be a comprehensive archive of publicly
released (versioned or dated) Scheme files.

That will take a non-trivial amount of disk space. I just downloaded the
full release backlog of a few of the bigger Scheme implementations
(using `wget --mirror` and FTP). This includes all source and binary
files made available by the distributors.

The sizes:

2.1G    ./bigloo
7.5G    ./gambit
222M    ./chicken
4.8G    ./mit
  15G    .

Git can handle repos this big, but they are not comfortable to use
casually. You have to be careful about where you can store forks and
clones, operations can be slow, etc.

(GitHub is believed to have a hard limit of 100G per repo; GitLab 10G.
Source:
https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com)

Hence, I will take the files themselves out of the current repo at
https://github.com/schemeorg/files.scheme.org and leave only the SHA
hashes there. I'll make a separate repo for the files. This means that
people wishing to add files to the archive don't have to clone a huge
repo onto their own computers and GitHub accounts. Instead they can
clone the small repo with the metadata, send a PR with the hashes and
URLs, and I can then download the file and add it to the big repo.

IMHO the big repo with the files should still be public so that people
can easily make backups of the whole file archive. But I expect that not
every contributor will want to do that.

Another question is where to find a server with tens of gigabytes
storage. Elf at least has a lot of disk space, so we could move
files.scheme.org to his server.