Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset Tool to add Timestamps #121

Merged
merged 19 commits into from
Oct 1, 2024
Merged

Dataset Tool to add Timestamps #121

merged 19 commits into from
Oct 1, 2024

Conversation

farzadab
Copy link
Contributor

@farzadab farzadab commented Sep 25, 2024

This PR adds a TimestampGeneration task to ds_tool.py. It uses Montreal Forced Aligner (MFA).

The task adds a new column (e.g. "timestamps") that has word-level alignments in the form of:

[ {"start": float in seconds, "end": float in seconds, "text": first word str}, ... ]

Usage example:

# Install Conda environment named aligner
# and download english_mfa acoustic model and dictionary
just install_mfa

# Run ds_tool (make sure upload name is not the same as input dataset_name)
just ds_tool timestamp -d fixie-ai/common_voice_17_0 -S en --upload_name fixie-ai/cv_ts  \
    -m english_mfa -T "\"{{text_proc.format_asr_text(sentence)}}\""

@farzadab farzadab marked this pull request as ready for review September 28, 2024 00:02
@farzadab
Copy link
Contributor Author

farzadab commented Oct 1, 2024

Actually if the changes here are okay I think I'll split ds_tool into multiple files, one per task, since this file is getting a bit too long.
I'll do it in the next PR.

@farzadab farzadab enabled auto-merge (squash) October 1, 2024 20:32
@farzadab farzadab merged commit d717b6e into main Oct 1, 2024
1 check passed
akshat0311 pushed a commit to jiviai/audio-llm that referenced this pull request Jan 30, 2025
* add timestamp forced aligner

* download conda commands

* faster TimestampGenerationTask
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants