Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Dask estimators serialization prior to training #6065

Merged

Conversation

viclafargue
Copy link
Contributor

Partially answers #6046

@viclafargue viclafargue requested a review from a team as a code owner September 10, 2024 08:31
@github-actions github-actions bot added the Cython / Python Cython or Python issue label Sep 10, 2024
Copy link
Member

@divyegala divyegala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a tiny test to make sure that the serialization and deserialization are both successful for un-trained models? Maybe also train the model after deserialization

@dantegd
Copy link
Member

dantegd commented Sep 25, 2024

@viclafargue seems like the test ran into an issue in the pytest in some jobs:

=================================== FAILURES ===================================
________________________ test_serialize_before_training ________________________

client = <Client: 'tcp://127.0.0.1:45687' processes=1 threads=1, memory=251.77 GiB>

    def test_serialize_before_training(client):
        X, y = make_regression(n_samples=1000, n_features=20, random_state=0)
        X, y = da.from_array(X), da.from_array(y)
    
        model = LinearRegression(client=client)
>       pickled_model = pickle.dumps(model)

test_dask_serialization.py:90: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <cuml.dask.linear_model.linear_regression.LinearRegression object at 0x7fca696b4510>

    def __getstate__(self):
>       internal_model = self._get_internal_model().result()
E       AttributeError: 'NoneType' object has no attribute 'result'

/opt/conda/envs/test/lib/python3.11/site-packages/cuml/dask/common/base.py:60: AttributeError

@viclafargue
Copy link
Contributor Author

@viclafargue seems like the test ran into an issue in the pytest in some jobs:

That's really strange, might possibly be missing something, but isn't there an issue with the CI?

@viclafargue viclafargue added bug Something isn't working non-breaking Non-breaking change labels Sep 27, 2024
@viclafargue viclafargue requested review from a team as code owners October 16, 2024 14:46
@github-actions github-actions bot added conda conda issue ci labels Oct 16, 2024
@viclafargue viclafargue changed the base branch from branch-24.10 to branch-24.12 October 16, 2024 14:46
@divyegala divyegala removed request for a team and KyleFromNVIDIA October 16, 2024 15:13
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 67.15%. Comparing base (ef7a61d) to head (39af9c2).

Files with missing lines Patch % Lines
python/cuml/cuml/dask/common/base.py 0.00% 3 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##           branch-24.12    #6065      +/-   ##
================================================
- Coverage         68.33%   67.15%   -1.18%     
================================================
  Files               193      193              
  Lines             12837    12839       +2     
================================================
- Hits               8772     8622     -150     
- Misses             4065     4217     +152     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@divyegala
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 417d980 into branch-24.12 Oct 17, 2024
63 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci conda conda issue Cython / Python Cython or Python issue non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants