From 3e234cb679daaa337cf0c037dcec891d6ad54823 Mon Sep 17 00:00:00 2001 From: Simon Devauchelle Date: Thu, 13 Jul 2023 15:08:43 +0200 Subject: [PATCH] README - update scoring table. --- README.md | 103 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 72 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 098fd9e..fd3489d 100644 --- a/README.md +++ b/README.md @@ -108,45 +108,86 @@ Class constructor accept 3 optional arguments: #### _VBx-based_ models Another gender segmentation system, which utilizes **x-vectors** (https://github.com/BUTSpeechFIT/VBx), is also available and can be specified in the class constructor for usage: -* vbx_based : (default:False): if set to True, performs gender segmentation using _vbx-based_ system on speech segment. Computation time is longer but gender detection can be better depending on your use (see scoring tables). +* vbx_based (default:False): if set to True, performs gender segmentation using _vbx-based_ system on speech segment. Computation time is longer but gender detection can be better depending on your use (see scoring tables). Warning : 'detect_gender' argument must be set to True. ### Gender detection scores **Frame-level evaluation** (collar = 500ms) -* on **ESTER** corpus - -| vbx_based | False | True | -|----------------------|:--------:|:------------:| -| **Female** recall | 96,88 | **98,97** | -| **Male** recall | 99,26 | **99,38** | -| **Female** precision | 97,53 | **97,54** | -| **Male** precision | 97,30 | **99,61** | -| F1 score | 97,74 | **98,87** | - -* on **REPERE** corpus - -| vbx_based | False | True | -|----------------------|:---------:|:---------:| -| **Female** recall | 95,86 | **97,00** | -| **Male** recall | **98,57** | 97,90 | -| **Female** precision | 90,05 | **94,95** | -| **Male** precision | 98,97 | **99,35** | -| F1 score | 95,82 | **98,87** | - -* **DATA-INA-FR** : a new annotated corpus of French audiovisual archives has been annotated. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ESTERREPEREDATA--INA-FR*
ISSVBx-basedISSVBx-basedISSVBx-based
RecallFemale96,8898,9795,8697,0096,1597,82
Male99,2699,3898,5797,9098,5595,60
PrecisionFemale97,5397,5490,0594,9595,0794,47
Male97,3099,6198,9799,3596,4796,85
F1-score97,7498,8795,8298,8796,5596,16
+ +\* **DATA-INA-FR** : a new annotated corpus of French audiovisual archives has been annotated. It represents 285 minutes of content from French television and radio channels. This corpus is much noisier, but more representative of the reality of an audiovisual stream. -We recommend setting vbx_based to False if you are processing such data. - -| vbx_based | False | True | -|----------------------|:---------:|:---------:| -| **Female** recall | 96,24 | **97,84** | -| **Male** recall | **97,66** | 94,66 | -| **Female** precision | **93,26** | 93,10 | -| **Male** precision | 96,38 | **97,01** | -| F1 score | **95,87** | 95,61 | +We recommend setting vbx_based to False if you are processing such data. ## Using _VBx-Based_ Voice Femininity Scoring