Skip to content

Proposal: parallelize in the compare method for improved efficiency #165

Open
@hlbotterman

Description

@hlbotterman

Description
The current implementation of the split processing is done sequentially, which can result in significant execution time, especially when dealing with large datasets or numerous splits or numerous imputers. In some cases, this sequential approach creates bottlenecks, leading to longer overall processing times than necessary.

Proposal
I propose introducing parallel processing for handling splits to improve efficiency and reduce the time required for the procedure. By utilizing joblib, we could take advantage of multi-core systems, allowing the splits to be processed concurrently.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions