Can Not Use Attention Maps Padding

[spacegoing]

I'm training 93x480p model with videos less than 93 frames, and need to pad frames / mask attention weights.

However,

Open-Sora-Plan/opensora/utils/dataset_utils.py

Lines 155 to 156 in 294993c

	if self.batch_size == 1 or self.group_frame or self.group_resolution:
	assert torch.all(attention_mask.bool())

As can be seen here, collate function assert no paddings in attention mask.

WOuld love to know why mask is asserted to be all True; And how do we pad attention mask if drop_short_ratio<1.0 when training with mixed lengths of clips.