Skip to content

Metadata.add_column can be slow #2125

Open
@amontanez24

Description

@amontanez24

Problem Description

The metadata.add_column method can sometimes be unreasonably slow (a few seconds). This seems to be because during its validation, we check if it isn't a faker function, which seems to be a slow method in RDT.

def _validate_sdtype(self, sdtype):
if not isinstance(sdtype, str):
raise InvalidMetadataError(
f'Invalid sdtype: {sdtype} is not a string. Please use one of the '
'supported SDV sdtypes.'
)
if sdtype in self._DEFAULT_SDTYPES:
return
if not is_faker_function(sdtype):
raise InvalidMetadataError(
f"Invalid sdtype: '{sdtype}' is not recognized. Please use one of the "
'supported SDV sdtypes.'
)

Expected behavior

  • See if the validation can be sped up. This may require speeding up the underlying function in RDT

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions