How to implement attention mechanism using sparse modules？

[Shaoyi0520]

Hello, I am working on a 2-dimensional UNet for sparse image denoising and I would like to integrate the attention mechanism (below) into the UNet network. But I did not succeed, if possible, I would like you to give me some examples of attention for sparse modules, thank you very much!

[bobonpu1]

能加个微信吗？

[btgraham]

https://github.com/PointsCoder/VOTR implements sparse attention; maybe that can be useful.
But what is the sparse 2D input? And does the sparsity pattern change with denoising?