Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Every scraping round ends with "context deadline exceeded" errors #1

Open
vgiotsas opened this issue Aug 24, 2020 · 0 comments
Open

Comments

@vgiotsas
Copy link

Hi @whyrusleeping

After updating the list of bootstrap nodes to a more recent one, I tried to run the DHT scrapper but I only recive context deadline exceeded errors. Please see a sample output below:

ubuntu@ip-172-31-57-88:~/work/ipfs-counter$ go run main.go
building new node to collect metrics with
starting new scrape round...
scraping.deadline...2020-08-24T16:49:59.396Z    ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.399Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.400Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.400Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.400Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.401Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.397Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.402Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.404Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.404Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:49:59.404Z       ERROR   dht_scrape      context deadline exceeded
.done!
updating stats took 144.022005ms
scraping...2020-08-24T16:50:39.674Z     ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:50:39.676Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:50:39.686Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:50:39.687Z       ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.681Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.682Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.682Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.682Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.688Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.688Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.689Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:50:39.690Z        ERROR   dht_scrape      context deadline exceeded
.........done!
updating stats took 223.59961ms
scraping...2020-08-24T16:51:20.061Z     ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:51:20.068Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:51:20.069Z       ERROR   dht_scrape      context deadline exceeded
.2020-08-24T16:51:20.070Z       ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.064Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.064Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.065Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.066Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.072Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.073Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.073Z        ERROR   dht_scrape      context deadline exceeded
2020-08-24T16:51:20.074Z        ERROR   dht_scrape      context deadline exceeded
.........done!
updating stats took 280.303022ms
scraping^Csignal: interrupt

I know the project may not be mainted anymore, but I just try different DHT crawlers to compile a list of working tools.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant