Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add audio spectrogram transformer, and full audio clip #406

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

add audio spectrogram transformer, and full audio clip #406

wants to merge 11 commits into from

Commits on Feb 3, 2023

  1. will use torchaudio for functions for waveform to spectrogram as well… 

    … as specaugment
    @lucidrains
    lucidrains committed Feb 3, 2023
    Configuration menu
    Copy the full SHA
    ef0c70a View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2023

  1. add audio spectrogram transformer that utilizes vision transformer (w… 

    …hich it is, as the premise is to change the waveform to a 2d "image" and patchify and attend)
    @lucidrains
    lucidrains committed Feb 4, 2023
    Configuration menu
    Copy the full SHA
    897be62 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. start moving all audio stuff into own modality file

    @lucidrains
    lucidrains committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    a3836ab View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into audio-compatible

    @lucidrains
    lucidrains authored Feb 6, 2023
    Configuration menu
    Copy the full SHA
    4e4fa67 View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2023

  1. decide whether to inherit from CLIP later, duplicate code for now

    @lucidrains
    lucidrains committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    a0cca92 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into audio-compatible

    @lucidrains
    lucidrains authored Feb 18, 2023
    Configuration menu
    Copy the full SHA
    c815a3e View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2023

  1. allow for passing in spectrogram, as well as being able to control wh… 

    …ether spectrogram is augmented or not
    @lucidrains
    lucidrains committed Feb 25, 2023
    Configuration menu
    Copy the full SHA
    68fa528 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2023

  1. allow for flexible number of channels, if custom audio spectrogram be… 

    …ing passed in
    @lucidrains
    lucidrains committed Mar 4, 2023
    Configuration menu
    Copy the full SHA
    bf0ce8b View commit details
    Browse the repository at this point in the history

  2. Revert "allow for flexible number of channels, if custom audio spectr… 

    …ogram being passed in"

This reverts commit bf0ce8b.
    @lucidrains
    lucidrains committed Mar 4, 2023
    Configuration menu
    Copy the full SHA
    40ad6f6 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2023

  1. handle input a bit more flexibly in audio transformer

    @lucidrains
    lucidrains committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    28ed83b View commit details
    Browse the repository at this point in the history

  2. whoops

    @lucidrains
    lucidrains committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    188944e View commit details
    Browse the repository at this point in the history