Bidirectional model?

[fransilvionGenomica]

Hello,

Thanks for the great paper! If I understand correctly, Mamba model is similar to a one directional LSTM. Is there a way to implement it in not causal but bidirectional way, so the model can see information from both sequence ends? It would be similar to BERT encoder architecture in that sense I guess.

[albertfgu]

There are many approaches to this. You can implement a variety of naive methods to incorporate information from both ends, such as in this PR: #52.

There are some slightly more clever methods that are in progress. Stay tuned!

[AxelElaldi]

Hi! Thank you for your awesome work! I am also interested in using Mamba with different bidirectional methods. Besides PR #52, is there any new features related to the more advanced methods you mentioned? Thanks!