Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(model) : add segmentation model based on self-supervised representation #1362

Merged
merged 25 commits into from
Sep 18, 2023

Commits on May 5, 2023

  1. add WaVLM-Base model to PyanNet.py in replacement of SincNet

    Added WavLM-Base model which replaces the SincNet feature extraction model
    within the PyanNet architecture (loaded outside of the class from
    HuggingFace.co).
    SevKod committed May 5, 2023
    Configuration menu
    Copy the full SHA
    6d3af2e View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. 2 Configuration menu
    Copy the full SHA
    d03906b View commit details
    Browse the repository at this point in the history

Commits on May 15, 2023

  1. add support of all Torchaudio self-supverised models to PyanNet, incl…

    …uding layer selection.
    
    Created a block (in replacement of the old WavLM one) called "selfsup.py" which loads and
    apply a specific SSL Torchaudio model, depending on PyanNet's input parameter. User can now
    also choose a specific layer which will then be used for feature extraction.
    Ex :
    
    seg_model = PyanNet(task=seg, model = "HUBERT_BASE", layer = 5)
    
    This will load "HUBERT_BASE" model and select the 6th layer for the feature extraction. If
    layer is not specified, will automatically use the first one (layer 0).
    
    All available models can be found at : https://pytorch.org/audio/main/pipelines.html
    SevKod committed May 15, 2023
    Configuration menu
    Copy the full SHA
    3fc2d37 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. add support of ssl models from huggingface to pyannote using PyanHugg…

    … class
    
    Can use pre-trained ssl models from huggingface using PyanHugg class.
    Tested (and working) models are :
    
         - "microsoft/wavlm-base"
         - "microsoft/wavlm-large"
         - "facebook/hubert-base-ls960"
         - "facebook/wav2vec2-base-960h"
    
    Class supports model and layer selection (as well as cache location for the
    downloaded model and configuration file).
    Ex :
    seg_model = PyanHugg(task=seg, selfsupervised={
    'model' : 'microsoft/wavlm-base',
    'layer' : 2,
    'cache' : 'mod_location/'})
    SevKod committed May 26, 2023
    Configuration menu
    Copy the full SHA
    1e370fc View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. Configuration menu
    Copy the full SHA
    e170eed View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2023

  1. Configuration menu
    Copy the full SHA
    9f81c30 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2023

  1. add support for fairseq pretrained ssl models

    Can load a fairseq ckpt from a pretrained model (which is converted to
    torchaudio wav2vec2 format)
    SevKod committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    e5330fc View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7a21fc9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6243f91 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. Configuration menu
    Copy the full SHA
    328505c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d4ddd53 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Configuration menu
    Copy the full SHA
    63a9e42 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Configuration menu
    Copy the full SHA
    cbd01a3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f608eb7 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Configuration menu
    Copy the full SHA
    d7e9203 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2023

  1. Configuration menu
    Copy the full SHA
    81aafdd View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Configuration menu
    Copy the full SHA
    b9c89b6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8aba20e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4a8bfe2 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2023

  1. Configuration menu
    Copy the full SHA
    2323105 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Configuration menu
    Copy the full SHA
    cedf042 View commit details
    Browse the repository at this point in the history
  2. chore: remove old PyanSup

    hbredin committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    06641bf View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Configuration menu
    Copy the full SHA
    31d08a4 View commit details
    Browse the repository at this point in the history
  2. doc: update changelog

    hbredin committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    421ba03 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Configuration menu
    Copy the full SHA
    5f9211c View commit details
    Browse the repository at this point in the history