You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
scrapy wouldn't create new request after i pause crawl and resume crawl, and redis wouldn't create spidername:requests set again, even i set SCHEDULER_PERSIST = True at settings.py.
#149
Open
Vickey-Wu opened this issue
Jul 24, 2019
· 0 comments
i set SCHEDULER_PERSIST = True at settings.py and try to run spider again, redis wouldn't create "spidername:dupefilter" redis set again, only dupefilter is still here.
The text was updated successfully, but these errors were encountered:
Vickey-Wu
changed the title
scrapy wouldn't crawl new request after i pause crawl and resume crawl, and redis wouldn't create redis:requests set again, even i set SCHEDULER_PERSIST = True at settings.py.
scrapy wouldn't create new request after i pause crawl and resume crawl, and redis wouldn't create spidername:requests set again, even i set SCHEDULER_PERSIST = True at settings.py.
Jul 24, 2019
pause crawl (ctrl+c) and resume crawl util 0 in redis:requests set, and when spider closed
spidername:requests
were deletedi set
SCHEDULER_PERSIST = True
at settings.py and try to run spider again, redis wouldn't create "spidername:dupefilter" redis set again, only dupefilter is still here.The text was updated successfully, but these errors were encountered: