Applicability of this method to object detection/mask prediction

[ameyskulkarni]

Hi @chihkuanyeh
Thanks for sharing the code of this method. The paper and the code are very helpful and understandable for the classification task in general. I wanted to ask if you have tried/thought about extending this work for object detectors/mask predictors?
If say, there is a two stage object detector, can we use representer values to somehow understand which training samples helped and which training samples harmed the box/mask prediction accuracy on the test image. If you haven't worked in this direction, do you see any major hurdle in implementing this method for an object detector? Is it even theoretically possible according to the Representer theorem?

Thank you

[chihkuanyeh]

Hi @ameyskulkarni, I can imagine this being done, but the difficulty would be that one would get a representer value for each pixel's prediction on each class. The simplest thing I can imagine is that for each class, sum (average) over all pixels with this specific class. Thus, for each different class in the image, one can obtain the most important training points that helped or hurt the prediction of this image of this class. There might be other difficulty that I did not realized.