This project contains a set of scripts used to scrape Ebay's products data using Scrapy Web Crawling Framework.
In the current stage, the list of products scraped is defined by a search string (the same used in eBay web page).
An example of the scraped data can be found in the data/ folder.
The image below shows a scraped data for the "iphone X 256gb" search string in ebay.com
)
You will need Python 3.x to run the scripts. Python can be downloaded here.
You have to install scrapy framework:
- In command prompt/Terminal: pip install scrapy
- If you are using Anaconda Python distribution: conda install -c conda-forge scrapy
Once you have installed scrapy framework, just clone/download this project, access the folder in command prompt/Terminal and run the following command:
scrapy crawl ebay -o products.csv
You can change the output format to JSON or XML by change the output file extension (ex: products.json).
The default search string is nintendo switch console and it can be changed in the command line with the -a flag. For example, to search to Xbox one X you can use:
scrapy crawl ebay -o products.csv -a search="Xbox one X"