Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Scraper Framework #68

Open
njall opened this issue Jun 28, 2019 · 0 comments
Open

New Scraper Framework #68

njall opened this issue Jun 28, 2019 · 0 comments
Assignees

Comments

@njall
Copy link
Contributor

njall commented Jun 28, 2019

Recreate the Ruby scraper library in NodeJS.

The motivation is to broaden the ability of 3rd parties to contribute scrapers by using a more ubiquitous language.

To recreate the scraper framework you'll need to make a new API client that interfaces with the write API. The Ruby version is kept as a separate library here: https://github.com/ElixirTeSS/TeSS_api_client

Then you'll have to produce a common utility library (see Scraper modules in https://github.com/ElixirTeSS/TeSS_scrapers/blob/master/lib/tess/scrapers/scraper.rb). In Ruby, the scrapers inherit from this module which interfaces with the TeSS API and collects statistics about success rates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants