Skip to content
This repository has been archived by the owner on Jul 15, 2024. It is now read-only.

More robust import error handling #12

Open
MalteJanz opened this issue Jul 2, 2024 · 1 comment
Open

More robust import error handling #12

MalteJanz opened this issue Jul 2, 2024 · 1 comment

Comments

@MalteJanz
Copy link
Owner

Currently if a sync API request fails for import:

  • It checks the error body for "data pointers"
  • Removes these invalid rows from the chunk and reports them to the user (with the line number in the CSV file)
  • Retries with another final request

There should also be a more general error handling if the above doesn't work:

  • write the entities one by one, also report errors to the user

See src/data/import.rs

@MalteJanz
Copy link
Owner Author

Also the current implementation may actually be wrong. There could be multiple errors for a single line and in that case it might actually remove some valid lines too 🙈

See:

fn remove_invalid_entries_from_chunk(

It stores to_be_removed in a Vec, so it could contain the same index twice. Maybe we should use a HashSet instead or something else to ensure the indices are unique

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant