Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SS task model is much more slower then SE why? #6

Open
MonolithFoundation opened this issue Dec 5, 2024 · 4 comments
Open

SS task model is much more slower then SE why? #6

MonolithFoundation opened this issue Dec 5, 2024 · 4 comments

Comments

@MonolithFoundation
Copy link

MonolithFoundation commented Dec 5, 2024

The SS task model is significantly slower than the SE model. Why is this the case?

When compared with the same mossformer-16k.

Also, for the SE task, mossformer-GAN is far superior to mossformer-se. However, it is much slower. Is there any way to have a model that is both fast and accurate?

(The better here means that GAN maintains the voice style without much distortion, but mossformer-se is very distorted.)

@alibabasglab
Copy link
Collaborator

Hi, thank you for your feedback. SS model is time-domain basis, the window size is quite small. That means it needs to process more frames compared to the time-frequency dimain se model.

@alibabasglab
Copy link
Collaborator

MossformerGAN is processing on both magnitude and phases, and trained using GAN. So it focuses more on perception, while mossformer-se is masking basis, and trained using mse losses

@MonolithFoundation
Copy link
Author

MossFormerGAN is a good more for noisy reduction, (as I noticed SS is hard to use since it's slow), but MossFormerGAN still slow on CPU.

Will there any further plan to make both model fast while also keeping the ability to denoise?

@alibabasglab
Copy link
Collaborator

Thank you for your feedback, we will be looking through ways to improve the inference speed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants