Boris Zarubin

[email protected]

B21-DS-01

Text Detoxification

Text Detoxification Task is a process of transforming the text with toxic style into the text with the same meaning but with neutral style.

Installation

git clone https://github.com/Poseidondon/text-detoxification
pip install -r requirements.txt

Data preprocessing

You can skip that part if you want to use pre-built vocab.

Python

from src.data.build_vocab import build_vocab
build_vocab()

CLI

cd src/data
python make_dataset.py

Inference

Python

from src.models.cond_BERT import load_condBERT
condBERT = load_condBERT()
toxic_example = "I like that f***ing show!"
print(condBERT(toxic_example))

CLI

cd src/models
python predict_model.py -f input.txt -o out.txt
# or
python predict_model.py -s "I like that f***ing show!"

Evaluation

CLI

cd src/models
python evaluation.py