Simple product crawler that gets name and url for all products on a webpage
To run, install all requirements with:
pip install -r requirements.txt
Then, run:
make run
The results will be stored on /data/products.csv, there will be a log on /data/log.
If the execution is interrupted, run it again and it will pick up where it left off.
You must have docker and docker-compose installed
to start container:
make docker