Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bad sitemap should not return 0 #253

Open
valentinedwv opened this issue Apr 1, 2024 · 1 comment
Open

bad sitemap should not return 0 #253

valentinedwv opened this issue Apr 1, 2024 · 1 comment
Assignees

Comments

@valentinedwv
Copy link
Member

valentinedwv commented Apr 1, 2024

While error was logged, the gleaner container returned success, causing scheduler/dagster failed to detect failure.

something wrong with neotomadb... no sitemap, returned HTML page.

"file":"/home/runner/work/gleaner/gleaner/internal/summoner/acquire/resources.go:134","func":"github.com/gleanerio/gleaner/internal/summoner/acquire.getSitemapURLList","level":"error","msg":"Error reading sitemap at:http://data.neotomadb.org/sitemap.xmlXML syntax error on line 9: attribute name without = in element","time":"2024-04-01T21:11:26Z"}
{"file":"/home/runner/work/gleaner/gleaner/internal/summoner/acquire/resources.go:75","func":"github.com/gleanerio/gleaner/internal/summoner/acquire.ResourceURLs","level":"error","msg":"Error getting sitemap urls for: neotomadbXML syntax error on line 9: attribute name without = in element","time":"2024-04-01T21:11:26Z"}



SourceStats:
  Start: 2024-04-01 21:11:26.856354745 +0000 UTC m=+0.339619903
  End: 2024-04-01 21:11:26.859096773 +0000 UTC m=+0.342361934
  Soruce:
    - name: neotomadb
      SitemapHttpError: 0 
      SitemapIssues: 0 
      SitemapSummoned: 0 
      SitemapCount: 0 
RunStats:
  Start: 2024-04-01 21:11:26.612979146 +0000 UTC m=+0.096244289
  Reason: Complete
  Soruce:
    - name: neotomadb
      Start: 2024-04-01 21:11:26.856354745 +0000 UTC m=+0.339619903
      End: 2024-04-01 21:11:26.85918183 +0000 UTC m=+0.342446984
      SitemapCount: 0 
      SitemapHttpError: 0 
      SitemapIssues: 0 
      SitemapSummoned: 0 
@valentinedwv
Copy link
Member Author

Hiting this with an r2r sitemap change. Dagster keeps running, because step completes fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants