Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test scaling to ½ billion works #95

Open
jonasob opened this issue Dec 16, 2014 · 0 comments
Open

Test scaling to ½ billion works #95

jonasob opened this issue Dec 16, 2014 · 0 comments
Assignees

Comments

@jonasob
Copy link
Member

jonasob commented Dec 16, 2014

Over time, the catalog should scale to be able to include upwards of ½ billion works, and quite a bit of work may be needed to get us there. We can look at this in a few different stages:

  • 50M works
  • 100M
  • 250M
  • 500M

We need to scale both mongodb and the hashdb, while retaining lookup speeds on both. Approaches to scale could include ripping auth/frontend work from the catalog to create a lookup-only endpoint API, which is more lightweight than the r/w endpoint API.

@jonasob jonasob self-assigned this Dec 16, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant