Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'FundaSpider' object has no attribute '_rules' #5

Open
jeroenvermunt opened this issue Dec 29, 2021 · 1 comment
Open

Comments

@jeroenvermunt
Copy link

When running the code after forking, I immediately got the following error:

AttributeError: 'FundaSpider' object has no attribute '_rules'

I solved it by adding the following line to init():

super(FundaSoldSpider, self).__init__(place=place)

The error stopped appearing, but now the following lines show up during scraping, giving an empty json file in the end.

[scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.funda.nl/koop/verkocht/eindhoven/p259/> (referer: None)
[scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.funda.nl/koop/verkocht/eindhoven/p268/> from <GET http://www.funda.nl/koop/verkocht/eindhoven/p268/>

there are the following possibilities:

  1. the fix solved the error but broke the scraper
  2. the fix solved the error, but the scraper does not work on a different aspect.

I'm am not sure which one it is, anyone have any idea on how to get the scraper to work?

@haydnjm
Copy link

haydnjm commented Mar 12, 2022

This fixes those problems + some others that prevent it from working, just in case you didn't figure it out yet! (only changes for the currently for-sale houses, not the sold ones)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants