Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

为什么运行以后out里什么都没有? #1

Open
shanyan-wcx opened this issue Mar 21, 2021 · 0 comments
Open

为什么运行以后out里什么都没有? #1

shanyan-wcx opened this issue Mar 21, 2021 · 0 comments

Comments

@shanyan-wcx
Copy link

G:\DaguerreSpider-master\Daguerre>scrapy crawl superspider
2021-03-21 14:17:09 [scrapy.utils.log] INFO: Scrapy 2.4.1 started (bot: Daguerre)
2021-03-21 14:17:09 [scrapy.utils.log] INFO: Versions: lxml 4.6.2.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 21.2.0, Python 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:18:16) [MSC v.1928 64 bit (AMD64)], pyOpenSSL 20.0.1 (OpenSSL 1.1.1j 16 Feb 2021), cryptography 3.4.6, Platform Windows-10-10.0.19041-SP0
2021-03-21 14:17:09 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2021-03-21 14:17:09 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'Daguerre',
'CONCURRENT_REQUESTS': 32,
'DOWNLOAD_DELAY': 1,
'NEWSPIDER_MODULE': 'Daguerre.spiders',
'ROBOTSTXT_OBEY': True,
'SPIDER_MODULES': ['Daguerre.spiders']}
2021-03-21 14:17:09 [scrapy.extensions.telnet] INFO: Telnet Password: 8853f12029b3d17c
2021-03-21 14:17:09 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.logstats.LogStats']
2021-03-21 14:17:10 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2021-03-21 14:17:10 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2021-03-21 14:17:11 [scrapy.middleware] INFO: Enabled item pipelines:
['Daguerre.pipelines.DaguerrePipeline']
2021-03-21 14:17:11 [scrapy.core.engine] INFO: Spider opened
2021-03-21 14:17:11 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2021-03-21 14:17:11 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2021-03-21 14:17:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://cl.286x.xyz/robots.txt> (referer: None)
2021-03-21 14:17:14 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://cl.286x.xyz/thread0806.php?fid=16&search=&page=1> (referer: None)
2021-03-21 14:17:14 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://cl.286x.xyz/thread0806.php?fid=16&search=&page=2> (referer: https://cl.286x.xyz/thread0806.php?fid=16&search=&page=1)
2021-03-21 14:17:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://cl.286x.xyz/thread0806.php?fid=16&search=&page=3> (referer: https://cl.286x.xyz/thread0806.php?fid=16&search=&page=2)
2021-03-21 14:17:16 [scrapy.core.engine] INFO: Closing spider (finished)
2021-03-21 14:17:16 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 1282,
'downloader/request_count': 4,
'downloader/request_method_count/GET': 4,
'downloader/response_bytes': 37408,
'downloader/response_count': 4,
'downloader/response_status_count/200': 4,
'elapsed_time_seconds': 5.242255,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2021, 3, 21, 6, 17, 16, 313344),
'log_count/DEBUG': 4,
'log_count/INFO': 10,
'request_depth_max': 2,
'response_received_count': 4,
'robotstxt/request_count': 1,
'robotstxt/response_count': 1,
'robotstxt/response_status_count/200': 1,
'scheduler/dequeued': 3,
'scheduler/dequeued/memory': 3,
'scheduler/enqueued': 3,
'scheduler/enqueued/memory': 3,
'start_time': datetime.datetime(2021, 3, 21, 6, 17, 11, 71089)}
2021-03-21 14:17:16 [scrapy.core.engine] INFO: Spider closed (finished)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant