feat: add Series|Expr.rank

[FBruzzesi]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Closes [Enh]: Support polars.Expr.rank #1323

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below.

This PR provides initial support for rank method. I will start it as a draft due to a bunch of shortcomings:

• pandas:
  • there is a (nasty) trick to make it work with nulls/nans and nullable dtypes (related: to BUG: rank does not respect na_option='keep' for numpy nullable integer dtypes, which should be fixed in pandas 3) - I will comment the trick in the code
  • support in over context is available
  • support in group_by context is not available.
• pyarrow:
  • does not support polars default method (namely, "average"), therefore if rank is called without specifying another method, it will end up raising an error
  • does not implement rank in a group by context
  • I am using pyarrow.compute.rank which is available but not exposed/documented (?)
• dask: it just does not support ranking
• polars:
  • group by context always returns an aggregate, in this case a list of ranks - which is fairly useless as it is a list of increasing/descreasing range until the size of the group

[FBruzzesi]

Here is the workaround.

@MarcoGorelli I was not able to properly use the pandas like util function get_dtype_backend to figure out the nullable backend. It should not really matter as the non-nullable backend would not result in integer type if the series contains nulls anyway

[FBruzzesi]

Hey @adamblake, this is an initial implementation to support rank. In the description I tried to explain all the shortcomings and the challenges I am facing.

[FBruzzesi]

I will need to come back to the failing tests: it seems that rank for pandas version 2.0.x with pyarrow backend fails for integer dtype but it works fine for floats

[MarcoGorelli]

Nice, thanks for sticking with this!

Does it risk being annoying if the default ('average') isn't available in several backends?

Could we not set any default, so the user is 'forced' to specify a method (like we do in quantile)?

[FBruzzesi]

Nice, thanks for sticking with this!

Of course! Adding the possibility to using it in over context is a great addition for the use case reported in the issue in the first place!

Does it risk being annoying if the default ('average') isn't available in several backends?

For now this is just pyarrow, right?

Could we not set any default, so the user is 'forced' to specify a method (like we do in quantile)?

I don't have a strong opinion on this, I am fine to give more responsibility to the user

[MarcoGorelli]

For now this is just pyarrow, right?

you're right sorry, i thought it would be the same for pyarrow-backed pandas. thinking ahead, I think duckdb doesn't support it either

[MarcoGorelli]

thanks @FBruzzesi !