Skip to content

Commit

Permalink
added README.md and RESULTS.md
Browse files Browse the repository at this point in the history
  • Loading branch information
JinZr committed Oct 20, 2024
1 parent 7eee6b9 commit 2831572
Show file tree
Hide file tree
Showing 3 changed files with 85 additions and 1 deletion.
26 changes: 26 additions & 0 deletions egs/libritts/ASR/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Introduction

LibriTTS is a multi-speaker English corpus of approximately 585 hours of read English speech at 24kHz sampling rate, prepared by Heiga Zen with the assistance of Google Speech and Google Brain team members.
The LibriTTS corpus is designed for TTS research. It is derived from the original materials (mp3 audio files from LibriVox and text files from Project Gutenberg) of the LibriSpeech corpus.
The main differences from the LibriSpeech corpus are listed below:
1. The audio files are at 24kHz sampling rate.
2. The speech is split at sentence breaks.
3. Both original and normalized texts are included.
4. Contextual information (e.g., neighbouring sentences) can be extracted.
5. Utterances with significant background noise are excluded.
For more information, refer to the paper "LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech", Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu, arXiv, 2019. If you use the LibriTTS corpus in your work, please cite this paper where it was introduced.


This recipe includes some different ASR models trained with [LibriTTS](https://openslr.org/60/).

[./RESULTS.md](./RESULTS.md) contains the latest results.

# Transducers

| | Encoder | Decoder |
|---------------------------------------|---------------------|--------------------|
| `zipformer` | Upgraded Zipformer | Embedding + Conv1d |

The decoder is modified from the paper
[Rnn-Transducer with Stateless Prediction Network](https://ieeexplore.ieee.org/document/9054419/).
We place an additional Conv1d layer right after the input embedding layer.
58 changes: 58 additions & 0 deletions egs/libritts/ASR/RESULTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Results

## zipformer (zipformer + pruned stateless transducer)

See <https://github.com/k2-fsa/icefall/pull/1746> for more details.

[zipformer](./zipformer)

### Non-streaming

#### normal-scaled model, number of model parameters: 65549011, i.e., 65.55 M

You can find a pretrained model, training logs, decoding logs, and decoding results at:
<https://huggingface.co/zrjin/icefall-asr-libritts-zipformer-2024-10-20>

You can use <https://github.com/k2-fsa/sherpa> to deploy it.

| decoding method | test-clean | test-other | comment |
|----------------------|------------|------------|--------------------|
| greedy_search | 2.83 | 5.91 | --epoch 30 --avg 5 |
| modified_beam_search | 2.80 | 5.87 | --epoch 30 --avg 5 |
| fast_beam_search | 2.87 | 5.86 | --epoch 30 --avg 5 |
| greedy_search | 2.76 | 5.68 | --epoch 40 --avg 16|
| modified_beam_search | 2.74 | 5.66 | --epoch 40 --avg 16|
| fast_beam_search | 2.75 | 5.67 | --epoch 40 --avg 16|
| greedy_search | 2.74 | 5.67 | --epoch 50 --avg 30|
| modified_beam_search | 2.73 | 5.58 | --epoch 50 --avg 30|
| fast_beam_search | 2.78 | 5.61 | --epoch 50 --avg 30|


The training command is:
```bash
export CUDA_VISIBLE_DEVICES="0,1"
./zipformer/train.py \
--world-size 2 \
--num-epochs 50 \
--start-epoch 1 \
--use-fp16 1 \
--exp-dir zipformer/exp \
--causal 0 \
--full-libri 1 \
--max-duration 3600
```
This was used on 2 Nvidia A800 GPUs, you'll need to adjust the `CUDA_VISIBLE_DEVICES`, `--world-size` and `--max-duration` according to your hardware.

The decoding command is:
```bash
export CUDA_VISIBLE_DEVICES="0"
for m in greedy_search modified_beam_search fast_beam_search; do
./zipformer/decode.py \
--epoch 50 \
--avg 30 \
--use-averaged-model 1 \
--exp-dir ./zipformer/exp \
--max-duration 600 \
--decoding-method $m
done
```
2 changes: 1 addition & 1 deletion egs/libritts/ASR/zipformer/asr_datamodule.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def add_arguments(cls, parser: argparse.ArgumentParser):
"--full-libri",
type=str2bool,
default=True,
help="""When enabled, use 960h LibriTTS.
help="""When enabled, use the entire LibriTTS training set.
Otherwise, use the 100h subset.""",
)

Expand Down

0 comments on commit 2831572

Please sign in to comment.