Added exception handling for UnicodeDecodeError #1533
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aayushkdev
force-pushed
the
issue-#1526
branch
2 times, most recently
from
a678664 to
02f9a7a
Compare
…e-org#1526 Signed-off-by: Aayush Kumar <[email protected]>
AyanSinhaMahapatra
requested changes
Apr 14, 2025
There was a problem hiding this comment.
@aayushkdev Thanks, but this is not the correct way to do this. See comments.
| return json.load(f) | ||
| except json.JSONDecodeError: | ||
| return | ||
| except UnicodeDecodeError as e: |
There was a problem hiding this comment.
This still creates an exception and this breaks the pipeline run, we need to catch and create a ProjectMessage https://github.com/aboutcode-org/scancode.io/blob/main/scanpipe/models.py#L1399 object to keep these in the /messages/ tab for that project, instead of raising an exception. Check similar invocations elsewhere for how to do this.
There was a problem hiding this comment.
Please run a d2d pipeline using the inputs at #1526 (comment), this should create an error message and the pipeline should complete successfully.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1526
Added exception handling for UnicodeDecodeError and added a catch all exception for load_json_from_file