ImportError: cannot import name 'login_to_hub' from 'doctr.models'

[javohirss]

Hello, I ran into an error when trying to train a model on a custom dataset:

could you help me with this pls

[felixdittrich92]

Hi @javohirss 👋,
seems you have no DL framework installed you can install it the following way:

!git clone https://github.com/mindee/doctr.git
%cd doctr
!pip install -e .[tf] --upgrade
!pip install -r references/requirements.txt

import doctr
print(doctr__version__)  <-- should be 0.5.2a0

[javohirss]

Hello, sorry, I have been installing DLF since yesterday (I chose tensorflow). Then I downloaded your project, went into it through PyCharm. I typed in the terminal those commands that I run in the collab and got the following error:

[felixdittrich92]

Hi 👋, on WIndows you have to follow these steps to install weasyprint: https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#windows

[javohirss]

It's me again )
Now with this error:

I know it's because of my letters (I'm using Tajik which differs from Russian by 5 characters), but how can I fix it?

[felixdittrich92]

Could you try to load the json outside of doctr and check if the same error raised ?

[felixdittrich92]

I think you should check that the default encoding in Windows is utf-8 or utf-16

[javohirss]

hi, i got same error when I running the json file outside of doctr. There I fixed it by adding utf8 encoding when opening:

and there is my Windows default encoding:

Should I add something to your code in this case or change the windows encoding? Btw, thx for paying attention to me)

[felixdittrich92]

yep only provide the model name

[javohirss]

It gaves me this:

[felixdittrich92]

ok then try python references\recognition\train_tensorflow.py crnn_vgg16_bn --font pass a ttf which is available in your Windows machine i think Arial.ttf is in every windows machine available
so do the following:
python references\recognition\train_tensorflow.py crnn_vgg16_bn --font Arial.ttf

[javohirss]

Same error, but instead of FreeMono.ttf there is Arial. Should I download some font and load it to some folder?

[felixdittrich92]

mh maybe try this: https://stackoverflow.com/questions/44358468/unable-to-use-windows-fonts-with-pil
or modify the training script: (pass None instead of fonts)

doctr/references/recognition/train_tensorflow.py

Line 168 in e538cc2

font_family=fonts,

doctr/references/recognition/train_tensorflow.py

Line 256 in e538cc2

font_family=fonts,

[javohirss]

Hi! This time I ran into this:

[felixdittrich92]

Hi @javohirss 👋,

python references\recognition\train_tensorflow.py crnn_vgg16_bn --font Arial.ttf was running well ?
I try to identify if anything with your instalation is wrong or with your data 🤔

Would also be good to share some information !
Run : https://github.com/mindee/doctr/blob/main/scripts/collect_env.py and post the output please :)

[javohirss]

I got this:

[javohirss]

and this, after running python references\recognition\train_tensorflow.py crnn_vgg16_bn --font Arial.ttf (I have set the None value to font field as you recommended):

[felixdittrich92]

mh really weird but what i see is that there is no NVIDIA driver displayed could you check this please ?

[javohirss]

Hi! I don't know why collect_env doesn't show the NVIDIA driver version. But I checked it in GeForce Experience:

[javohirss]

I tried with python 3.8.12 and still got:

[felixdittrich92]

Mh yeah still some issues with multiprocessing on Windows :/
Replacing fastprogress with tqdm and setting num_workers=0 (dataloader) should help.. but if possible switch to Linux is the better alternative 😅

[javohirss]

Hello! I switched to Linux and ran the code:
python references\recognition\train_pytorch.py crnn_vgg16_bn --font FreeMono.ttf
And it finally worked!
Then I added tajik lang (it differs from Russian in the presence of several characters) to vocabs.py:

and got error:

[felixdittrich92]

Hi @javohirss 👋 ,
good news 👍
This says that there are characters in your dataset which are not present in the given vocab please verify this :)

For example one label: 'xyz' and you provide as vocab: 'xy'

[javohirss]

Hello! At first, it really turned out that there were characters in the dataset that were not contained in the vocab. I wrote a function that checks each label against a vocab to see if the label's characters are in the vocab. I have corrected all wrong characters but still getting this error. Is there any way to find exactly where something is going wrong? I mean that wrong label or character.

[felixdittrich92]

Hi copy the function:

doctr/doctr/datasets/utils.py

Line 62 in 3d7c812

def encode_string(

and test against your labels.json with a short script so you can easily find the samples which raise the error :)

[felixdittrich92]

Hi @javohirss 👋,
you will need lots of data to do this (english and tajik samples)
and you need to modify the training script:

NOTE: while inference you have to provide the vocab in the same way

from collections import OrderedDict
vocab = ''.join(OrderedDict.fromkeys(VOCABS['tajik'] + VOCABS['english'])    

About the training i need more details about the logs (loss / exact / partial) and maybe 1 or 2 samples from your dataset

[javohirss]

Logs:

First I tested on passport image (that's what I need in my project) and got bad results, but I thought that it was in the presence of English, and I used an image consisting entirely of Tajik. Result is also bad :

My dataset consists of 20k train images and 5k val. Examples:

Maybe it's all about fonts?

[felixdittrich92]

Mh yes, to compare the pretrained versions was trained on ~10M real world word crops.
If you train the model from scratch you need much more data (with different fonts, backgrounds, etc.).
I would also suggest (if possible) to use as real world dataset to validate the model (so in short train it with synthetic data and use a much smaller dataset in --val_path then the metrics are much more meaningful)

[javohirss]

Ahh.. it's Tajik. I am pretty sure that there is no such dataset 😅
Theoretically, I could create it myself 🙊 , but how many images do we need? I mean for validation
And is the given script in the generator folder for data synthesis?

[felixdittrich92]

Hi @javohirss 👋,
Yeah unfortunately i think too :/
You have to test a bit with it but i would say for real word crops a range between 5k - 10k should be possible and a good starting point.
Our word generator needs some improvements but sure you could test it.
Example:
python3 /home/user/Desktop/doctr/references/recognition/train_pytorch.py crnn_vgg16_bn --train-samples 10000 --val-samples 1000 --min-chars 1 --max-chars 25 --font xyz.ttf

change:

doctr/references/recognition/train_pytorch.py

Line 185 in acb9f64

vocab = VOCABS[args.vocab]

to:

from collections import OrderedDict
vocab = ''.join(OrderedDict.fromkeys(VOCABS['tajik'] + VOCABS['english'])  

This creates automatically a synthetic dataset with the generator and triggers the training
ATTENTION: you need to install fonts which are compatible with tajik and english in your case
more information about installing custom fonts on your machine:
https://github.com/mindee/doctr/tree/main/references/recognition#using-custom-fonts

Offtopic: https://github.com/clovaai/synthtiger could also be a very good option to create a dataset

If you have the possibility to use your GPU for the inference and latency is of secondary importance, you could also try our MASTER implementation, i.e. look at the model. This generalizes much faster to less data. I was able to achieve good results with it even with toy experiments.

Cheers ✌️