Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

evaluate whether we want to stop suppressing errors by default in _process_one_query method of QueryCollection #520

Closed
DaniBodor opened this issue Nov 6, 2023 · 4 comments · Fixed by #636
Assignees
Labels
stale issue not touched from too much time

Comments

@DaniBodor
Copy link
Collaborator

In principle, we expect users to have "good" PDB files. Currently, we are catching errors in case there is a problem with the pdbs. Evaluate whether if all PDBs are good, errors still occur and whether we prefer by default raising them and having an option to suppress, or by default catching them and only raising a warning (as is the situation now)

Copy link

github-actions bot commented Dec 8, 2023

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale issue not touched from too much time label Dec 8, 2023
@DaniBodor DaniBodor self-assigned this Jul 9, 2024
@DaniBodor
Copy link
Collaborator Author

The decision is to by default throw an error if a faulty PDB file is encountered, meaning the code will stop executing at this point. We will also implement a flag to allow users to suppress the error and proceed with only the non-faulty data.

Potential issues with this approach:

  • if running on a large dataset and only a small % of data is faulty, then time/resources are wasted by re-running the processing step if only one of the last queries is faulty.
  • deal with duplicate files (probably the old files are just overwritten by the new ones)
    • we can try and see if it's possible to (optionally) skip queries for which data exists, which would solve both problems above
  • when suppressing, we should send a clear log of the number of times the error would have occurred, and a list of all instances.

@gcroci2 gcroci2 removed the stale issue not touched from too much time label Jul 9, 2024
Copy link

github-actions bot commented Aug 9, 2024

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale issue not touched from too much time label Aug 9, 2024
@gcroci2 gcroci2 linked a pull request Sep 4, 2024 that will close this issue
@DaniBodor
Copy link
Collaborator Author

We still suppress the errors, but provide better feedback of what is going on. No option was added to enforce hard fails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale issue not touched from too much time
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants