Fix to_pandas writable bug for datetime and timedelta types

[galipremsagar]

Description

Fixes: #17743

This PR fixes writable flag for numpy arrays. This bug is only specific to these two types.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[mroeschke]

Curious how #17743 as a bug affects subsequent operations (i.e. is this a bug we really need to worry about?)

pandas.Index is not supposed to be publicly mutable, so the underlying numpy array not being writable doesn't seem that big of a deal, especially since the fix requires making a CPU copy of the data

[galipremsagar]

Curious how #17743 as a bug affects subsequent operations (i.e. is this a bug we really need to worry about?)

pandas.Index is not supposed to be publicly mutable, so the underlying numpy array not being writable doesn't seem that big of a deal, especially since the fix requires making a CPU copy of the data

It's not serious. But AFAIK it is because ._data yields a DatetimeArray and that is being mutated in pandas pytests leading to failures in the test suite. If not this fix, we will need to try to address this for cudf.pandas case at least. But since users can run into this issue with cudf classic aswell I went ahead and opened a short-term fix until the arrow bug is fixed.

[mroeschke]

But since users can run into this issue with cudf classic

Yeah so I'm curious what subsequent issue a user would run into if a pandas.Index returned from to_pandas() if the underlying numpy array was not writable. (i.e. if no public APIs break when using this Index I'm just tempted to stay this is a won't fix to avoid the CPU copy)

[galipremsagar]

But since users can run into this issue with cudf classic

Yeah so I'm curious what subsequent issue a user would run into if a pandas.Index returned from to_pandas() if the underlying numpy array was not writable. (i.e. if no public APIs break when using this Index I'm just tempted to stay this is a won't fix to avoid the CPU copy)

This is the user-facing case I'm talking about:

In [1]: import pandas as pd
i 
In [2]: i = pd.Index([1, 2, 3], dtype="datetime64[ns]")

In [4]: i.to_numpy()
Out[4]: 
array(['1970-01-01T00:00:00.000000001', '1970-01-01T00:00:00.000000002',
       '1970-01-01T00:00:00.000000003'], dtype='datetime64[ns]')

In [5]: x = i.to_numpy()

In [6]: x[0] = 1

In [7]: x
Out[7]: 
array(['1970-01-01T00:00:00.000000001', '1970-01-01T00:00:00.000000002',
       '1970-01-01T00:00:00.000000003'], dtype='datetime64[ns]')

In [8]: x[0] = 2

In [9]: x
Out[9]: 
array(['1970-01-01T00:00:00.000000002', '1970-01-01T00:00:00.000000002',
       '1970-01-01T00:00:00.000000003'], dtype='datetime64[ns]')

In [10]: import cudf

In [11]: i = cudf.Index([1, 2, 3], dtype="datetime64[ns]")

In [13]: x = i.to_pandas().to_numpy()

In [14]: x
Out[14]: 
array(['1970-01-01T00:00:00.000000001', '1970-01-01T00:00:00.000000002',
       '1970-01-01T00:00:00.000000003'], dtype='datetime64[ns]')

In [15]: x[0] = 2
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[15], line 1
----> 1 x[0] = 2

ValueError: assignment destination is read-only

[mroeschke]

Could we do something like

if not (arrow_type and nullable):
    return pd.Index(self.data_array_view(mode="read").copy_to_host())
return super().to_pandas(nullable=nullable, arrow_type=arrow_type)

instead? I think that should avoid the extra CPU copy.

[galipremsagar]

It would but this doesn't support nulls:

[left]:  TimedeltaIndex([NaT, NaT, NaT, NaT, NaT, NaT, NaT], dtype='timedelta64[ns]', freq=None)
[right]: TimedeltaIndex(['0 days 00:00:00.000000012', '0 days 00:00:00.000000011',
                '0 days 00:00:00.000000002', '0 days 00:00:00.000002234',
                '0 days 00:00:00.000002343', '0 days 00:00:00.000023432',
                '0 days 00:00:00.000023234'],
               dtype='timedelta64[ns]', freq=None)

[mroeschke]

Could we just do

gi = cudf.Index(...).to_numpy()
pi = pd.Index(...).to_numpy()

assert gi.flags == pi.flags

instead?

[galipremsagar]

the ownable flag is not the same for both so we will not be able to this assertion.

[mroeschke]

Ah OK. Darn yeah I forgot about to_numpy. Left some other feedback on the implementation

[galipremsagar]

/merge