Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of non-UTF-8 encoded files in validator #63

Open
bleonar5 opened this issue Aug 15, 2024 · 0 comments
Open

Improve handling of non-UTF-8 encoded files in validator #63

bleonar5 opened this issue Aug 15, 2024 · 0 comments
Assignees
Labels
Scoping Discussion regarding the scope, design, and features of the psych-DS validator

Comments

@bleonar5
Copy link
Contributor

Description

The validator currently throws errors when it encounters non-UTF-8 encoded files. This behavior interrupts the validation process and prevents the user from receiving a complete report of all issues with their dataset. Additionally, the validator is checking the encoding of all files, including those that are not directly relevant to the psych-DS schema.

Current Behavior

  • The validator throws an error and stops processing when it encounters a non-UTF-8 encoded file.
  • All files in the dataset are being checked for UTF-8 encoding, regardless of their relevance to the psych-DS schema.

Desired Behavior

  • When encountering a non-UTF-8 encoded file, the validator should:
    • Handle the error gracefully without interrupting the overall validation process.
    • Add an issue to the validation output indicating that the file is not UTF-8 encoded.
  • The validator should only check the encoding of relevant files:
    • Canonical datafiles (e.g., CSV files specified in the dataset's metadata).
    • JSON metadata files.
    • Other file types specified as required by the psych-DS schema.
@bleonar5 bleonar5 added the Scoping Discussion regarding the scope, design, and features of the psych-DS validator label Aug 15, 2024
@bleonar5 bleonar5 self-assigned this Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scoping Discussion regarding the scope, design, and features of the psych-DS validator
Projects
None yet
Development

No branches or pull requests

1 participant