Size of files.scheme.org Lassi Kortela (06 Jul 2022 11:10 UTC)
Re: Size of files.scheme.org John Cowan (06 Jul 2022 13:07 UTC)
Hosting platforms Lassi Kortela (06 Jul 2022 13:32 UTC)
Re: Size of files.scheme.org elf (06 Jul 2022 22:51 UTC)
Re: Size of files.scheme.org Lassi Kortela (09 Jul 2022 15:49 UTC)
Re: Size of files.scheme.org Lassi Kortela (09 Jul 2022 15:57 UTC)
Re: Size of files.scheme.org John Cowan (10 Jul 2022 02:09 UTC)
Re: Size of files.scheme.org elf (06 Jul 2022 22:54 UTC)

Re: Size of files.scheme.org elf 06 Jul 2022 22:53 UTC
Sorry, I don't know why the reply cut out everything. Previous email
still stands.

On 06/07/2022 16:06, John Cowan wrote:
> I use S3 for backing store and have for many years.  I recommend it.
> You have to pay, but Amazon doesn't pull the plug on you without
> warning.  I currently have 186 GB and I pay about US $4 per month.
>
> On Wed, Jul 6, 2022 at 7:10 AM Lassi Kortela <xxxxxx@lassi.io
> <mailto:xxxxxx@lassi.io>> wrote:
>
>     files.scheme.org <http://files.scheme.org> should one day be a
>     comprehensive archive of publicly
>     released (versioned or dated) Scheme files.
>
>     That will take a non-trivial amount of disk space. I just downloaded
>     the
>     full release backlog of a few of the bigger Scheme implementations
>     (using `wget --mirror` and FTP). This includes all source and binary
>     files made available by the distributors.
>
>     The sizes:
>
>     2.1G    ./bigloo
>     7.5G    ./gambit
>     222M    ./chicken
>     4.8G    ./mit
>        15G    .
>
>     Git can handle repos this big, but they are not comfortable to use
>     casually. You have to be careful about where you can store forks and
>     clones, operations can be slow, etc.
>
>     (GitHub is believed to have a hard limit of 100G per repo; GitLab 10G.
>     Source:
>     https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com
>     <https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com>)
>
>     Hence, I will take the files themselves out of the current repo at
>     https://github.com/schemeorg/files.scheme.org
>     <https://github.com/schemeorg/files.scheme.org> and leave only the SHA
>     hashes there. I'll make a separate repo for the files. This means that
>     people wishing to add files to the archive don't have to clone a huge
>     repo onto their own computers and GitHub accounts. Instead they can
>     clone the small repo with the metadata, send a PR with the hashes and
>     URLs, and I can then download the file and add it to the big repo.
>
>     IMHO the big repo with the files should still be public so that people
>     can easily make backups of the whole file archive. But I expect that
>     not
>     every contributor will want to do that.
>
>     Another question is where to find a server with tens of gigabytes
>     storage. Elf at least has a lot of disk space, so we could move
>     files.scheme.org <http://files.scheme.org> to his server.
>