Re: Size of files.scheme.org
elf 06 Jul 2022 22:53 UTC
Sorry, I don't know why the reply cut out everything. Previous email
still stands.
On 06/07/2022 16:06, John Cowan wrote:
> I use S3 for backing store and have for many years. I recommend it.
> You have to pay, but Amazon doesn't pull the plug on you without
> warning. I currently have 186 GB and I pay about US $4 per month.
>
> On Wed, Jul 6, 2022 at 7:10 AM Lassi Kortela <xxxxxx@lassi.io
> <mailto:xxxxxx@lassi.io>> wrote:
>
> files.scheme.org <http://files.scheme.org> should one day be a
> comprehensive archive of publicly
> released (versioned or dated) Scheme files.
>
> That will take a non-trivial amount of disk space. I just downloaded
> the
> full release backlog of a few of the bigger Scheme implementations
> (using `wget --mirror` and FTP). This includes all source and binary
> files made available by the distributors.
>
> The sizes:
>
> 2.1G ./bigloo
> 7.5G ./gambit
> 222M ./chicken
> 4.8G ./mit
> 15G .
>
> Git can handle repos this big, but they are not comfortable to use
> casually. You have to be careful about where you can store forks and
> clones, operations can be slow, etc.
>
> (GitHub is believed to have a hard limit of 100G per repo; GitLab 10G.
> Source:
> https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com
> <https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com>)
>
> Hence, I will take the files themselves out of the current repo at
> https://github.com/schemeorg/files.scheme.org
> <https://github.com/schemeorg/files.scheme.org> and leave only the SHA
> hashes there. I'll make a separate repo for the files. This means that
> people wishing to add files to the archive don't have to clone a huge
> repo onto their own computers and GitHub accounts. Instead they can
> clone the small repo with the metadata, send a PR with the hashes and
> URLs, and I can then download the file and add it to the big repo.
>
> IMHO the big repo with the files should still be public so that people
> can easily make backups of the whole file archive. But I expect that
> not
> every contributor will want to do that.
>
> Another question is where to find a server with tens of gigabytes
> storage. Elf at least has a lot of disk space, so we could move
> files.scheme.org <http://files.scheme.org> to his server.
>