New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate improving scalability of the nanopub pipeline #21

Open

Ostrzyciel opened this issue Nov 24, 2023 · 0 comments

Labels

discussion enhancement

Milestone

Member

Ostrzyciel commented Nov 24, 2023

The pipeline currently caches the individual nanopubs, but not the merge result. We should also cache it.
The merging is done in rdflib, which will get slow as the dataset expands (see Nanopublications dataset – future scope #19). We could replace it with, e.g., Apache Jena RIOT in CLI mode... or Apache Jena Fuseki, operated via HTTP calls.

Ostrzyciel added the discussion label

Ostrzyciel mentioned this issue

Display nanopub dataset statistics in the docs #22

Open

Ostrzyciel added the enhancement label

Ostrzyciel added this to the 1.1.0 milestone

Ostrzyciel modified the milestones: 1.1.0, 1.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment