Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

之前没有问题,突然拉取不到数据~ #147

Open
penndu opened this issue Jul 2, 2024 · 8 comments
Open

之前没有问题,突然拉取不到数据~ #147

penndu opened this issue Jul 2, 2024 · 8 comments
Assignees

Comments

@penndu
Copy link

penndu commented Jul 2, 2024

Run python ./hexo_circle_of_friends/run.py
2024-07-02 16:12:13|WARNING|2009|py.warnings|/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/scrapy/utils/request.py:254: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.

It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.

See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
  return cls(crawler)

2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|Initialization complete
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.spiders.hexo_circle_of_friends|起始url: https://dusays.com/friends/
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|----------------------
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|友链总数 : 0
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|失联友链数 : 0
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|共 506 篇文章
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|最后运行于:2024-07-02 16:12:13
2024-07-02 16:12:14|INFO|2009|hexo_circle_of_friends.pipelines.sql_pipe|done!
@CCKNBC CCKNBC pinned this issue Jul 2, 2024
@CCKNBC
Copy link
Member

CCKNBC commented Jul 2, 2024

弱小无助, breaking changes 了属于是,果然能跑就不要动

@CCKNBC
Copy link
Member

CCKNBC commented Jul 2, 2024

暂时回滚了 Actions 的依赖,只升级了出错的依赖,并新开了分支
https://github.com/Rock-Candy-Tea/hexo-circle-of-friends/tree/147-no-data

@CCKNBC CCKNBC self-assigned this Jul 2, 2024
@AGEN233
Copy link

AGEN233 commented Jul 3, 2024

而且vecerl部署的话,app/all会提示This Serverless Function has crashed.

@YL2209
Copy link

YL2209 commented Jul 11, 2024

5.0.8 版本的基础之上,把两个 requirements.txt 文件的 uvicorn 升级到 0.30.1,其他不用改。

@xyhcode
Copy link
Contributor

xyhcode commented Jul 13, 2024

5.0.8 版本的基础之上,把两个 requirements.txt 文件的 uvicorn 升级到 0.30.1,其他不用改。

改了还是
image

@Akimio521
Copy link

Akimio521 commented Jul 21, 2024

同样的问题,使用主分支还是分支147-no-data都无法解决,现在请教一下应该如何解决

@AGEN233
Copy link

AGEN233 commented Jul 24, 2024

同样的问题,使用主分支还是分支147-no-data都无法解决,现在请教一下应该如何解决

#149

@Akimio521
Copy link

同样的问题,使用主分支还是分支147-no-data都无法解决,现在请教一下应该如何解决

#149

我们遇到的问题似乎不同,这是我的详细情况#153

  1. 我看#149中主要说的是爬取数据后将数据保存至sqlite有问题,那么爬取部分应该是正常的
  2. 我是无法正常爬去友链,未到爬去部分
  3. 我使用的是mongodb作为我的数据库,是一个典型的nosql数据库,修改SQLAlchemy应该不影响吧(不太确定对mongodb的数据库操作中是否用到相关函数)

@CCKNBC CCKNBC closed this as completed Jul 27, 2024
@CCKNBC CCKNBC reopened this Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants