Summarization model for short texts based on pure transformer model with bpe encoding.
- Python 3.5 or higher
- PyTorch 0.4.0 or higher
- sentencepiece
- gensim
- tensorboardX
First of all, each part of dataset must stores in common folder and .tsv files, e.g. ./dataset/train.tsv and ./dataset/test.tsv (./dataset/sample.tsv by default uses for sampling summarizations). Each line of the document has two parts: source and target separated by tab symbol. Tabular files don't need headers. Preprocessor build vocabulary and train word embeddings.
For preprocess use:
$ python preprocess.py
For train model use
$ python train.py --cuda --pretrain_emb
After training model is saving into ./models_dumps/ directory.
You can tune model with lots arguments available in model. If you want a good result, use much deeper configuration of the model! Default configuration is used for tests.
For generate summarizations use
$ python sample.py --inp=sample_part --out=output_file.txt
Where sample_part is dataset part (e.g. ./dataset/sample_part.tsv).
See help of each module for more information and available arguments.