Skip to content

Latest commit

 

History

History
37 lines (26 loc) · 1.56 KB

README.md

File metadata and controls

37 lines (26 loc) · 1.56 KB
Raw

Opensource Data

Data source for opensource frontend. This repo scrape data hourly and publish it as JSON files.

Usage

Data sources are available at: developersdo.github.io/opensource-data.

Development

You will need to:

  1. Clone this repo.
  2. Install dependencies: npm i
  3. Run migrations: npm run migrations:run
  4. Copy .env.default to .env.
  5. Create a personal GitHub access token (with no scopes) and add the value to OSD_GH_TOKEN in your .env.
    • Note: You can skip that if you don't need to run the scraper locally.

Scripts

Script Description
npm run migrations:run Run all migrations.
npm run migrations:add [name] Add a new migration.
npm run scrape Scrape all data.
npm run scrape -- --only=users Scrape users data only.
npm run scrape -- --only=repos Scrape repos data only.
npm run generate Generate public site and JSON files.

Debugging

Prepend any commmand with DEBUG=osd:*. For example:

DEBUG=osd:* npm run scrape