Integration with sklearn

[blasern]

Hi

Do you plan to add support for integration with sklearn, to use e.g. cross-validation for bandwidth selection? Would be great to do something like this with KDEpy.

This would need the estimator.get_params(), etc. functions required from sklearn.

Ideally there would also be a estimator.score() function.

[tommyod]

I'm not against it at all.

I guess we would need to evaluate on arbitrary grid points, in order to evaluate a score on the data. But FFTKDE generates equidistant grid points and throws away the original data. This would mean doing high-dimensional interpolation. I haven't looked into how hard or easy that might be, or if there is some other clever way of doing it.

[blasern]

Agree that the scoring function may be non-trivial. But adding all other functionality would allow users to write their own scoring functions, which may be reasonable for some cases.

[tommyod]

Agreed. The sklearn API is very nice in it's own right, and a lot of people are ac customized to it and to some degree expect estimators to follow the API. I imitated it, but the APIs differ a little bit.

I'll leave the issue open. Up for grabs for anyone. I might look into it when (if?) I have time in the future.

[syrte]

Evaluating the density of arbitrary points would be absolutely a nice feature.
You mentioned "high-dimensional interpolation on equidistant grids". To my knowledge, one easy and fast solution is to use scipy.ndimage.map_coordinates.
FYI, in case it might help, I have written a wrapper to this function for my own use which is available at https://github.com/syrte/handy/blob/master/interpolate.py

[inti-abbate]

Hi
I just sent a PR with a proposal for a solution to this issue, based on a grid_search_cv function which is a simplified version of sklearn's GridSearchCV class. Since FFTKDE doesn't allow evaluating on a arbitrary grid, the method only works for NaiveKDE and TreeKDE. Of course the performance won't be as good, but for my work this is good enough, and of course you can first optimize BW with a TreeKDE and the use it to create a FFTKDE.

In the past I tried implementing an integration directly with sklearn, but I found 2 problems:

• sklearn doesn't takes into account test weights. This issue was pointed out and is being discussed here, so possibly will be fixed some day.
• By the way sklearn makes cross validation, it can't work with variable bandwidths. As I understood, they first clone the model, which would copy the full variable BW, and then fit it with each train fold, wich is a subset of the original dataset, resulting in models with more bandwidths than data points.

The first problem could be tolerated, but I couldn't think of a solution for the second problem, and doesn't look like sklearn will provide a solution. My conclusion was that the most reasonable solution was to implement the cross validation by hand (using only numpy). It isn't that hard if you restrict the model you pass to the grid search: instead of implementing a get_params method, I just clone a KDE model with model.__class__(model.kernel, model.bw, model.norm) (or replace model.bw by each BW of the grid).

[adrinjalali]

might be of interest: scikit-learn/scikit-learn#26896

[jonas-eschle]

Evaluating the density of arbitrary points would be absolutely a nice feature. You mentioned "high-dimensional interpolation on equidistant grids". To my knowledge, one easy and fast solution is to use scipy.ndimage.map_coordinates. FYI, in case it might help, I have written a wrapper to this function for my own use which is available at syrte/handy@master/interpolate.py

Thanks for this push and the amazingly fast library! I would double down on this and claim that the evaluation on arbitrary points is the only use-case (apart of pretty plots).

But FFTKDE generates equidistant grid points and throws away the original data.

I fail to see what this has to do with the evaluation? We don't need this at all, the evaluation should happen on new, arbitrary points whose values are determined by the interpolation of the grid points that the KDE holds, AFAIU, right?