Getting Started

Ready to contribute? Here's how to set up scrapy-redis for local development.

Setup environment

1. Fork the scrapy-redis repo on GitHub.

2. Clone your fork locally:

git clone [email protected]:your_name_here/scrapy-redis.git

3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:

pip install virtualenv==20.0.23
virtualenv --python=/usr/bin/python3 ~/scrapy_redis
source ~/scrapy_redis/bin/activate
cd scrapy-redis/
pip install -r requirements-install.txt
pip install .

4. Create a branch for local development:

git checkout -b name-of-your-bugfix-or-feature

Now you can make your changes locally.

Setup testing environment

1. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:

pip install -U tox
tox

6. Note that if the error of No module named scrapy_redis shows, please check the install scrapy-redis of your branch by:

pip install .

7. Or change the import lines:

from scrapy_redis import xxx # from this
from src.scrapy_redis import xxx # to this

8. Commit your changes and push your branch to GitHub:

git add .
git commit -m "Your detailed description of your changes."
git push origin name-of-your-bugfix-or-feature

9. Submit a pull request through the GitHub website.