Skip to content

Commit

Permalink
refactor: change base64 encoding/decoding registry naming (#1184)
Browse files Browse the repository at this point in the history
* refactor: base64 encoding/decoding registry renaming

* docs: fix mistakes in nemo documentation
  • Loading branch information
IgnatovFedor authored Apr 20, 2020
1 parent 804de52 commit fdab459
Show file tree
Hide file tree
Showing 5 changed files with 13 additions and 13 deletions.
4 changes: 2 additions & 2 deletions deeppavlov/configs/nemo/asr_tts.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"in": "speech_in_encoded",
"pipe": [
{
"class_name": "base64_encode_bytesIO",
"class_name": "base64_decode_bytesIO",
"in": ["speech_in_encoded"],
"out": ["speech_in"]
},
Expand All @@ -22,7 +22,7 @@
"out": ["speech_out"]
},
{
"class_name": "bytesIO_decode_base64",
"class_name": "bytesIO_encode_base64",
"in": ["speech_out"],
"out": ["speech_out_encoded"]
}
Expand Down
4 changes: 2 additions & 2 deletions deeppavlov/core/common/registry.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"api_router": "deeppavlov.models.api_requester.api_router:ApiRouter",
"basic_classification_iterator": "deeppavlov.dataset_iterators.basic_classification_iterator:BasicClassificationDatasetIterator",
"basic_classification_reader": "deeppavlov.dataset_readers.basic_classification_reader:BasicClassificationDatasetReader",
"base64_encode_bytesIO": "deeppavlov.models.nemo.common:ascii_to_bytes_io",
"base64_decode_bytesIO": "deeppavlov.models.nemo.common:ascii_to_bytes_io",
"bert_as_summarizer": "deeppavlov.models.bert.bert_as_summarizer:BertAsSummarizer",
"bert_classifier": "deeppavlov.models.bert.bert_classifier:BertClassifierModel",
"bert_ner_preprocessor": "deeppavlov.models.preprocessors.bert_preprocessor:BertNerPreprocessor",
Expand All @@ -23,7 +23,7 @@
"bilstm_gru_nn": "deeppavlov.models.ranking.bilstm_gru_siamese_network:BiLSTMGRUSiameseNetwork",
"bilstm_nn": "deeppavlov.models.ranking.bilstm_siamese_network:BiLSTMSiameseNetwork",
"bow": "deeppavlov.models.embedders.bow_embedder:BoWEmbedder",
"bytesIO_decode_base64": "deeppavlov.models.nemo.common:bytes_io_to_ascii",
"bytesIO_encode_base64": "deeppavlov.models.nemo.common:bytes_io_to_ascii",
"capitalization_featurizer": "deeppavlov.models.preprocessors.capitalization:CapitalizationPreprocessor",
"char_splitter": "deeppavlov.models.preprocessors.char_splitter:CharSplitter",
"char_splitting_lowercase_preprocessor": "deeppavlov.models.preprocessors.capitalization:CharSplittingLowercasePreprocessor",
Expand Down
4 changes: 2 additions & 2 deletions deeppavlov/models/nemo/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
log = getLogger(__name__)


@register('base64_encode_bytesIO')
@register('base64_decode_bytesIO')
def ascii_to_bytes_io(batch: Union[str, list]) -> Union[BytesIO, list]:
"""Recursively searches for strings in the input batch and converts them into the base64-encoded bytes wrapped in
Binary I/O objects.
Expand All @@ -50,7 +50,7 @@ def ascii_to_bytes_io(batch: Union[str, list]) -> Union[BytesIO, list]:
return list(map(ascii_to_bytes_io, batch))


@register('bytesIO_decode_base64')
@register('bytesIO_encode_base64')
def bytes_io_to_ascii(batch: Union[BytesIO, list]) -> Union[str, list]:
"""Recursively searches for Binary I/O objects in the input batch and converts them into ASCII-strings.
Expand Down
10 changes: 5 additions & 5 deletions docs/features/models/nemo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ above. ``labels`` is a list of characters used in TTS model training.
Audio encoding end decoding.
----------------------------

:class:`~deeppavlov.models.nemo.common.ascii_to_bytes_io` and :class:`~deeppavlov.models.nemo.common.bytes_io_to_ascii`
was added to the library to achieve uniformity at work with both text and audio data. Classes can be used to encode
:func:`~deeppavlov.models.nemo.common.ascii_to_bytes_io` and :func:`~deeppavlov.models.nemo.common.bytes_io_to_ascii`
was added to the library to achieve uniformity at work with both text and audio data. Components can be used to encode
binary data to ascii string and decode back.

Quck Start
Expand All @@ -58,9 +58,9 @@ Install requirements and download model files.
python -m deeppavlov install asr_tts
python -m deeppavlov download asr_tts
Examples below use `soundservice <https://python-sounddevice.readthedocs.io/en/0.3.15/index.html>`_ library. Install
it with ``pip install soundservice==0.3.15``. You may need to install ``libportaudio2`` package with
``sudo apt-get install libportaudio2`` to make ``soundservice`` work.
Examples below use `sounddevice <https://python-sounddevice.readthedocs.io/en/0.3.15/index.html>`_ library. Install
it with ``pip install sounddevice==0.3.15``. You may need to install ``libportaudio2`` package with
``sudo apt-get install libportaudio2`` to make ``sounddevice`` work.

.. note::
ASR reads and TTS generates single channel WAV files. Files transferred to ASR are resampled to the frequency
Expand Down
4 changes: 2 additions & 2 deletions tests/test_configs/nemo/tts2asr_test.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@
"out": ["speech"]
},
{
"class_name": "bytesIO_decode_base64",
"class_name": "bytesIO_encode_base64",
"in": ["speech"],
"out": ["ascii"]
},
{
"class_name": "base64_encode_bytesIO",
"class_name": "base64_decode_bytesIO",
"in": ["ascii"],
"out": ["speech_restored"]
},
Expand Down

0 comments on commit fdab459

Please sign in to comment.