This repository has been archived by the owner on Jul 8, 2018. It is now read-only.
Releases: crawlerclub/x
Releases · crawlerclub/x
Release v0.1.0
Features
- Focused crawler based on configuration of xpath rules.
- Crawler configuration editor, a web based json editor.
- Concurrency crawling by goroutines.
- Multiple crawlers support, each crawler has its own priority.
- Crawled items could be processed further by arbitrary javascript codes.
- Crawled items could be saved to configured ElasticSearch server automatically.
- Support incremental crawling.
- Dashboard for monitoring system loads.