Python web crawler tool. scrapes internal and external urls
Clone this repo:
git clone https://github.com/rlyonheart/tarantula.git
cd tarantula
pip install -r requirements.txt
| Flag | MetaVar | Usage |
|---|---|---|
-c/--count |
NUMBER | Number of links to crawl (default is 30) |
-v/--verbose |
run tarantula in verbose mode |