Models

Models

The toolkit comes with a number of models, both shallow and deep. The ls-models tool is used to show the available models and the details of the configuration for each. For example,

% ls-models
C:\dev\sounds\video-tutorial>ls-models
Loading aisp properties from file:/c:/dev/aisp/aisp.properties
cnn
cnn-delta-acc
dcase
gmm
knn
lpknn
mvi
neural-net
normal-dist-anomaly

This is the list of models available in toolkit and the names of models that can be used in the other tools. For example,

% train -model gmm ...

Each model, including feature extraction and model specifics, is defined using JavaScript. The details of the model definition can be seen by specifying the name of the model. For example,

% ls-models gmm
Warning: Nashorn engine is planned to be removed from a future JDK release
Loading aisp properties from file:/c:/dev/aisp/aisp.properties
// gmm JavaScript /org/eng/aisp/classifier/factory/gmm.jsyt
    // Model parameters
    var numGaussians=8
    var useDiagonalCovariance=true

    // Feature parameters
    var fe;
    var fp = null;
    var resamplingRate=44100
    var minFreq=20
    var maxFreq=20000
    var windowSizeMsec=40
    var windowShiftMsec=windowSizeMsec * 50/100.0
    var featureLen=64
    var transform=null
    var useDiskCache=false
    var featureExtractor='MFCC'
    var normalizeProcessor='false'
    var deltaProcessor='false'
    // Define the feature extractor
    if (featureExtractor === 'FFT') {
      fe = new FFTFeatureExtractor(resamplingRate, minFreq, maxFreq, false, true, featureLen)
    } else if (featureExtractor === 'LogMel') {
        fe = new LogMelFeatureExtractor(resamplingRate, featureLen, minFreq, maxFreq, 0)
    } else {    // MFCC
      fe = new MFCCFeatureExtractor(resamplingRate, featureLen, minFreq, maxFreq, featureLen);
    }
    // Define the feature processor
    if (normalizeProcessor === 'true')
        fp = new NormalizingFeatureProcessor(true,true,true,true)
    if (deltaProcessor === 'true')  {
      var dfp = new DeltaFeatureProcessor(2,[1,1,1]);
        if (fp != null)
            fp = new PipelinedFeatureProcessor(fp,dfp)
        else
            fp = dfp
    }
    // Build the feature gram extractor
    var fge = new FeatureGramDescriptor(windowSizeMsec, windowShiftMsec, fe, fp);

    // Finally, create the classifier
    var classifier = new GMMClassifier(transform, fge, useDiskCache,
                        numGaussians, useDiagonalCovariance, GMMClassifier.DEFAULT_UNKNOWN_THRESH_COEFF);

This defines a number of key aspects of feature extraction in addition to model-specific parameters. This capability enables you to customize your models by capturing this into your own JavaScript model file and then using this file in training and evaluation. For example

% ls-models gmm > mygmm.js
% vi mygmm.js     # Make changes to the model definition and save
% train -model mygmm.js ...