Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surface more detailed error info when detecting metadata from dataframes #2327

Open
npatki opened this issue Dec 20, 2024 · 0 comments
Open
Labels
feature request Request for a new feature

Comments

@npatki
Copy link
Contributor

npatki commented Dec 20, 2024

Problem Description

This problem was first observed in #2323.

I may have a pandas DataFrame containing data that is not in the form that SDV expects. For example, I may have a column of dtype object in which the column values are dictionaries. This is not really supported from SDV right now, as it only models data that is int, float, strings, etc.

In such a case, the detect_from_dataframes function crashes with an error such as:

TypeError: unhashable type: 'dict'

This isn't really helpful in telling me which particular column or table is having an issue.

Expected behavior

As we detect metadata from a DataFrame, column by column, it would be useful to wrap the detection code in a try/catch statement. That way, if there is an error, we can print out the table/column name (in addition to the original error).

Eg.

InvalidDataError: Unable to detect metadata for table 'users' column 'age' due to an invalid data format.

TypeError: unhashable type: 'dict'

It would be nice if the original stack trace were also provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

1 participant