ruby_crawl

This repo contains some crawlers implemented in Ruby.

Usage

First fire up the VersionEye backend services like described here.

source ./scripts/set_vars_for_dev.sh

gem install bundler
bundle install

rake console
VersioneyeCore.new
NugetCrawler.crawl

docker-compose down

For commercial support send a message to [email protected].

This repository is licensed under the AGPL-3.0 license. Send a message to [email protected] if you are interested in a commercial license.

Name	Name	Last commit message	Last commit date
Latest commit reiz Update strategy for packagist Feb 5, 2020 ac72ed6 · Feb 5, 2020 History 906 Commits
config	config	Change MongoDB config	Nov 12, 2017
data	data	add custom CodeplexLicenseCrawler; rename bsd-x to bsd-x-clause;	Feb 14, 2017
lib	lib	Update strategy for packagist	Feb 5, 2020
log	log	Add log to git	Jul 3, 2014
scripts	scripts	Update tests	Jul 10, 2017
spec	spec	Merge branch 'master' into update_npm_crawler	Jul 26, 2017
tasks	tasks	Remove unwanted repos	Jan 27, 2020
.document	.document	Initial commit to ruby_crawl.	May 25, 2014
.gitignore	.gitignore	add licence matcher by url; tuned li filter for AWS pages;	Jul 27, 2016
.ruby-gemset	.ruby-gemset	Add .ruby-gemset for automation	Nov 27, 2015
.ruby-version	.ruby-version	Remove unwanted repos	Jan 27, 2020
Dockerfile	Dockerfile	Optimize Dockerfile	Jan 27, 2020
Gemfile	Gemfile	Remove unwanted repos	Jan 27, 2020
Gemfile.lock	Gemfile.lock	Remove unwanted repos	Jan 27, 2020
LICENSE.txt	LICENSE.txt	Update license to AGPL	Jul 10, 2017
README.md	README.md	Add license note	Jul 10, 2017
Rakefile	Rakefile	Fix rake console	Jan 31, 2017
VERSION	VERSION	Version bump to 0.1.0	May 25, 2014
circle.yml	circle.yml	Update Core.	Jul 10, 2017
ruby_crawl.gemspec	ruby_crawl.gemspec	Update logger format.	Jan 5, 2016
supervisord.conf	supervisord.conf	Docerise this project.	Mar 6, 2016
untitled	untitled	Update Core	Jul 20, 2017