Obtaining feature importances from attention maps

[sachinvs7]

Hi
Do you have code examples to showcase section 5.3 in the Revisiting Deep Learning Models for Tabular Data paper? - "Obtaining feature importances from attention maps."

I'm implementing the FT-Transformer architecture and would like to derive/plot both attention maps and IG on a dataset. Couldn't find any examples of such methods for tabular deep learning online. Appreciate your guidance. Thanks !

[Yura52]

Hi! Regarding our method for obtaining the feature importance maps, I recommend taking a look at this discussion: #2

As for IG, I don't remember for sure what we used to compute it, but I suspect we used the Captum library.

Does this help?

[sachinvs7]

Thanks for getting back!. Also wanted to confirm if "rtdl_revisiting_models" is indeed the main package to focus on.
In discussion #2; n_features can include categorical ones too?

[Yura52]

Hi!

• Yes, rtdl_revisiting_models is the recommended implementation of the models from the paper.
• Yes, I think the code in Feature importance results #2 allows using categorical features as well.