Support faster processing using pandas or polars functions in IterableDataset.map()

[lhoestq]

Allow super fast processing using pandas or polars functions in IterableDataset.map() by adding support to pandas and polars formatting in IterableDataset

import polars as pl
from datasets import Dataset

ds = Dataset.from_dict({"i": range(10)}).to_iterable_dataset()
ds = ds.with_format("polars")
ds = ds.map(lambda df: df.with_columns(pl.col("i"), pl.col("i").add(1).alias("i+1")), batched=True)
ds = ds.with_format(None)
print(next(iter(ds)))
# {'i': 0, 'i+1': 1}

It leverages arrow's zero-copy features from/to pandas and polars.

related to #3444 #6762

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.