No improvement in OpenSora runtimes when using the experimetn scripts.

[nighting0le01]

What are the values for threshold, window_size and n_calib to use for opensora and other video models? the ones in the
experiments/opensora_480p.sh and experiments/opensora.sh give no improvemnt at all

[asahni04]

the paper shows reduction in flops by upto 40% but same i don't see any improvement in runtime or flops with the .sh scripts in experiments for opensora. please guide

[asahni04]

@hahnyuan any suggestions? what loss thresholds and n_calibs to run to see the imporvement in runtime for opensora?

[asahni04]

@Probe100 any suggestions?

[nighting0le01]

Xuefei Ning @Probe100 @hahnyuan no i can't see the ablations and numbers for opensora in the paper and the code seems to assign full_attention at all thresholds for opensora? is there some bug ? can you please confirm the settings used for opensora??

[Probe100]

@nighting0le01 @asahni04 The bug is related to forward_with_cfg function in opensora/schedulers/iddpm/init.py (In OpenSora repo). DiTFastAttn implements a pre-hook for calibration purposes. The forward_with_cfg function in OpenSora was calling model.forward() directly instead of model() This direct call bypassed the pre-hook, preventing proper calibration. We directly modified the source code of OpenSora at the very beginning so the bug was not found until we tested our code on a new environment. We just updated our code and that should fix the bug. Please clear your cache before running the code again to ensure the fix takes effect.

[nighting0le01]

yeah i did find it and updated it but thanks for updating the main codebase too!

[asahni04]

@Probe100 thanks i was able to find the bug and fix it but can be please suggest the optimal params for opensora? uptill which there was not much drop in quality

[nighting0le01]

yeah @Probe100 could you please share the best settings? why is window_size 50 when it is supposed to be 1/8 hidden size?i get broken results if i set 0.025 0..05,0.075,0.1 and so on with window_size 50 for opensora

[asahni04]

@Probe100 yeah quality becomes very bad for anything more than 0.02 threshold. i'm currently trying with larger window sizes but not that hopefull . trying with 512,256,128 for 0.03, 0.04. as a loss threshold lower than this gives single digit percentage speedup :( . would appreciate any tips @Probe100