Size of files.scheme.org
Lassi Kortela
(06 Jul 2022 11:10 UTC)
|
Re: Size of files.scheme.org
John Cowan
(06 Jul 2022 13:06 UTC)
|
Hosting platforms
Lassi Kortela
(06 Jul 2022 13:32 UTC)
|
Re: Size of files.scheme.org
elf
(06 Jul 2022 22:51 UTC)
|
Re: Size of files.scheme.org
Lassi Kortela
(09 Jul 2022 15:49 UTC)
|
Re: Size of files.scheme.org
Lassi Kortela
(09 Jul 2022 15:57 UTC)
|
Re: Size of files.scheme.org
John Cowan
(10 Jul 2022 02:09 UTC)
|
Re: Size of files.scheme.org elf (06 Jul 2022 22:53 UTC)
|
Re: Size of files.scheme.org elf 06 Jul 2022 22:53 UTC
Sorry, I don't know why the reply cut out everything. Previous email still stands. On 06/07/2022 16:06, John Cowan wrote: > I use S3 for backing store and have for many years. I recommend it. > You have to pay, but Amazon doesn't pull the plug on you without > warning. I currently have 186 GB and I pay about US $4 per month. > > On Wed, Jul 6, 2022 at 7:10 AM Lassi Kortela <xxxxxx@lassi.io > <mailto:xxxxxx@lassi.io>> wrote: > > files.scheme.org <http://files.scheme.org> should one day be a > comprehensive archive of publicly > released (versioned or dated) Scheme files. > > That will take a non-trivial amount of disk space. I just downloaded > the > full release backlog of a few of the bigger Scheme implementations > (using `wget --mirror` and FTP). This includes all source and binary > files made available by the distributors. > > The sizes: > > 2.1G ./bigloo > 7.5G ./gambit > 222M ./chicken > 4.8G ./mit > 15G . > > Git can handle repos this big, but they are not comfortable to use > casually. You have to be careful about where you can store forks and > clones, operations can be slow, etc. > > (GitHub is believed to have a hard limit of 100G per repo; GitLab 10G. > Source: > https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com > <https://stackoverflow.com/questions/38768454/repository-size-limits-for-github-com>) > > Hence, I will take the files themselves out of the current repo at > https://github.com/schemeorg/files.scheme.org > <https://github.com/schemeorg/files.scheme.org> and leave only the SHA > hashes there. I'll make a separate repo for the files. This means that > people wishing to add files to the archive don't have to clone a huge > repo onto their own computers and GitHub accounts. Instead they can > clone the small repo with the metadata, send a PR with the hashes and > URLs, and I can then download the file and add it to the big repo. > > IMHO the big repo with the files should still be public so that people > can easily make backups of the whole file archive. But I expect that > not > every contributor will want to do that. > > Another question is where to find a server with tens of gigabytes > storage. Elf at least has a lot of disk space, so we could move > files.scheme.org <http://files.scheme.org> to his server. >