Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resilience for shard sources #601

Open
straight-shoota opened this issue Dec 2, 2023 · 0 comments
Open

Resilience for shard sources #601

straight-shoota opened this issue Dec 2, 2023 · 0 comments

Comments

@straight-shoota
Copy link
Member

Shards is decentral by design and thus systematically resiliant against a single point of failure.
In practice, most publicly available shards are hosted on GitHub, which thus acts as a single part of failure for a big portion of the ecosystem. So there's quite a big point of failure for that. One might argue that GitHub is too big too fail, and maybe there's some truth to that. But even GitHub has had some temporal outages in the past.

Regardless of GitHub's prominent role: any hoster can temporally (or permanently) drop out. If that happens, shards install won't be able to fetch shards hosted there. If the shard source is still available somewhere else, you could still make it work somehow, for example with manual overrides to the dependency location in shard.yml or shard.override.yml.
This requires explict interaction and knowledge about mirror repositories. So it's not ideal.

It's generally a good idea to have libraries available on multiple hosters. This improves resilience against provider losses.

Maybe we could find a way to include information about repository mirros in the shard metadata?
I was thinking about the repository property of shard.yml: It could allow multiple values, pointing to alternative sources for the shard. shards could use that information when fetching the dependency.

Of course, this is a bit of a problem because when you don't have the dependency installed yet, shards could't know that its shard.yml announces alternative sources.
The information would be available when the dependency is already installed (e.g. for shards update).
We could also consider to store such metadata about dependencies in a separte location that's checked into source control of the dependee. Maybe shard.lock could be enhanced to hold this kind of information?

And of course with a structured definition for repository metadata, shard indices such as https://shardbox.org or https://shards.info could make it publicly available which would survive the loss of a single hoster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant