Name		Name	Last commit message	Last commit date
parent directory ..
Cargo.toml		Cargo.toml
README.md		README.md
callback.rs		callback.rs
debug.rs		debug.rs
download.rs		download.rs
download_to_react.rs		download_to_react.rs
example.rs		example.rs
scrape.rs		scrape.rs
serde.rs		serde.rs

README.md

Examples

First git clone https://github.com/spider-rs/spider.git and cd spider.

Basic

Simple concurrent crawl Simple.

cargo run --example example

Live handle index example Callback.

cargo run --example callback

Enable log output Debug.

cargo run --example debug

Scrape the webpage with and gather html Scrape.

cargo run --example scrape

Scrape and download the html file to fs Download HTML. *Note: Only HTML is downloaded.

cargo run --example download

Scrape and download html to react components and store to fs Download to React Component.

cargo run --example download_to_react

Crawl the page and output the links via Serde.

cargo run --example serde --features serde