Skip to content

The Simple project to separate mixed voice. Using "Permutation Invariant Training Loss" and "PairWise Neg SisDr Loss"

Notifications You must be signed in to change notification settings

vuthede/speech_separation_PIT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Speech Separation

The simple project to separate mixed voice (2 clean voices) to 2 separate voices.

Result Example (Clisk to hear the voices): mix || prediction voice1 || prediction voice2

Mix Spectrogram

mix

Predict Voice1's Spectrogram

pred1

Predict Voice2's Spectrogram

pred2

1. Quick train

Step 1:

Download LibriMixSmall, extract it and move it to the root of the project.

Step 2:

./train.sh

It will take about ONLY 2-3 HOURS to train with normal GPU. After each epoch, the prediction is generated to ./viz_outout folder.

2. Quick inference

./inference.sh The result will be generated to ./viz_outout folder.

3. More detail

4. Current problem

Due to small dataset size for fast training, the model is a bit overfitting to the training set. Use the bigger dataset will potentially help to overcome that. Some suggestions:

  1. Use the original LibriMix Dataset which is way much bigger (around 60 times bigger that what I have trained).
  2. Use this work to download much more in-the-wild dataset and use datasets/VoiceMixtureDataset.py instead of the Libri one that I am using. p/s I have trained and it work too.

About

The Simple project to separate mixed voice. Using "Permutation Invariant Training Loss" and "PairWise Neg SisDr Loss"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published