You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I suggest you read the paper "On Scalar Embedding of Relative Positions in Attention Models". In that paper, they explain the implemented bucketing function.
Hey @lucidrains, thanks for keeping these models implemented. In line 88
video-diffusion-pytorch/video_diffusion_pytorch/video_diffusion_pytorch.py
Lines 84 to 88 in f55f1b0
max_exact
as the half ofnum_buckets
, whose value was already halved in line 84.I think that is duplicated and should be changed to identity:
The text was updated successfully, but these errors were encountered: