Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When downloading raw data (harvesting) capture errors we encounter #40

Closed
odscjames opened this issue Jun 25, 2020 · 4 comments
Closed

When downloading raw data (harvesting) capture errors we encounter #40

odscjames opened this issue Jun 25, 2020 · 4 comments
Assignees
@odscjames

Comments

@odscjames
Copy link
Collaborator

No description provided.

@odscjames
Copy link
Collaborator Author

odscjames commented Jun 25, 2020 •
edited
Loading

New table: `download_raw_errors'

  • URL - URL error occurred on
  • Error - the error
  • At - time error occurred
  • Publisher Feed - which feed we were working on when this happened

We can hold a lot of data in table; I'd suggest just starting with logging all errors. We can add something later to clean out old errors automatically

@odscjames odscjames self-assigned this Jun 25, 2020
@odscjames
Copy link
Collaborator Author

Also see #21

@odscjames odscjames mentioned this issue Jun 25, 2020
Open
@robredpath
Copy link
Collaborator

Capture errors during harvesting

odscjames added a commit that referenced this issue Jul 16, 2020
@odscjames
download-raw: Store errors
d11ff9b 
#40
@odscjames odscjames mentioned this issue Jul 16, 2020
Merged
odscjames added a commit that referenced this issue Jul 16, 2020
@odscjames
download-raw: Store errors
92fd450 
#40
@odscjames
Copy link
Collaborator Author

Table and writing to it done. (Stuff on #21 not done, we'll deal with that there)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@odscjames odscjames

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@robredpath @odscjames