Adding a new character to the vocab throws error

[ep0p]

While training the recognition with my dataset i encounter issues while i am trying to add new characters that are not in the french vocab like '«»' (this one is already in the arabic vocab) or 'm²'.

This is how i am adding the chars:
VOCABS["french"] = VOCABS["english"] + "àâéèêëîïôùûüçÀÂÉÈÊËÎÏÔÙÛÜÇ" + "»«m²"

The error shown is:
File "/home/epop/dataset_prep/doctr/references/recognition/train_pytorch.py", line 487, in <module> main(args) File "/home/epop/dataset_prep/doctr/references/recognition/train_pytorch.py", line 233, in main model = recognition.__dict__[args.arch](pretrained=args.pretrained, vocab=vocab) File "/home/epop/dataset_prep/doctr/doctr/models/recognition/crnn/pytorch.py", line 283, in crnn_vgg16_bn return _crnn("crnn_vgg16_bn", pretrained, vgg16_bn_r, ignore_keys=["linear.weight", "linear.bias"], **kwargs) File "/home/epop/dataset_prep/doctr/doctr/models/recognition/crnn/pytorch.py", line 243, in _crnn feat_extractor = backbone_fn(pretrained=pretrained_backbone).features # type: ignore[call-arg] File "/home/epop/dataset_preparring/doctr/doctr/models/classification/vgg/pytorch.py", line 88, in vgg16_bn_r return _vgg( File "/home/epop/dataset_prep/doctr/doctr/models/classification/vgg/pytorch.py", line 61, in _vgg load_pretrained_params(model, default_cfgs[arch]["url"], ignore_keys=_ignore_keys) File "/home/epop/dataset_prep/doctr/doctr/models/utils/pytorch.py", line 70, in load_pretrained_params model.load_state_dict(state_dict) File "/home/epop/anaconda3/envs/ds3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2189, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for VGG: size mismatch for classifier.weight: copying a param with shape torch.Size([126, 512]) from checkpoint, the shape in current model is torch.Size([128, 512]). size mismatch for classifier.bias: copying a param with shape torch.Size([126]) from checkpoint, the shape in current model is torch.Size([128]).

As far as i understand it comes from a pretrained model which was trained on size 126 and now me adding those characters increases the size.

What to do in this case?

[felixdittrich92]

Hi @ep0p 👋,

Best would be create your own vocab for example:

VOCABS["frenchplus"] = VOCABS["french"] + "»«m²"

Then start the training with your new vocab:
--pretrained --vocab="frenchplus"

Don't miss later on to pass the same vocab if you load your trained model.

Best regards,
Felix

[ep0p]

Hi @felixdittrich92 ,

It's me again :)
I have done what you suggested, and another issue popped up:

python references/recognition/train_pytorch.py crnn_vgg16_bn --train_path ../ds_small_rec/train --val_path ../ds_small_rec/val --epochs 5 --pretrained --vocab="frenchplus" --device 0
Namespace(arch='crnn_vgg16_bn', train_path='../ds_small_rec/train', val_path='../ds_small_rec/val', train_samples=1000, val_samples=20, font='FreeMono.ttf,FreeSans.ttf,FreeSerif.ttf', min_chars=1, max_chars=12, name=None, epochs=5, batch_size=64, device=0, input_size=32, lr=0.001, weight_decay=0, workers=None, resume=None, vocab='frenchplus', test_only=False, freeze_backbone=False, show_samples=False, wb=False, push_to_hub=False, pretrained=True, sched='cosine', amp=False, find_lr=False, early_stop=False, early_stop_epochs=5, early_stop_delta=0.01)
Validation set loaded in 0.2703s (40197 samples in 629 batches)
Train set loaded in 0.485s (160786 samples in 2512 batches)
Training loss: 1.16361:   7%|███████▊                                                                                                     | 180/2512 [00:11<02:24, 16.14it/s]
Traceback (most recent call last):██▊                                                                                                     | 179/2512 [00:11<01:57, 19.91it/s]
  File "/home/epop/dataset_prep/doctr/references/recognition/train_pytorch.py", line 487, in <module>
    main(args)
  File "/home/epop/dataset_prep/doctr/references/recognition/train_pytorch.py", line 392, in main
    fit_one_epoch(model, train_loader, batch_transforms, optimizer, scheduler, amp=args.amp)
  File "/home/epop/dataset_prep/doctr/references/recognition/train_pytorch.py", line 122, in fit_one_epoch
    train_loss = model(images, targets)["loss"]
  File "/home/epop/anaconda3/envs/ds3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/epop/anaconda3/envs/ds3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/epop/dataset_prep/doctr/doctr/models/recognition/crnn/pytorch.py", line 227, in forward
    out["loss"] = self.compute_loss(logits, target)
  File "/home/epop/dataset_prep/doctr/doctr/models/recognition/crnn/pytorch.py", line 185, in compute_loss
    ctc_loss = F.ctc_loss(
  File "/home/epop/anaconda3/envs/ds3.9/lib/python3.9/site-packages/torch/nn/functional.py", line 2687, in ctc_loss
    return torch.ctc_loss(
RuntimeError: Expected tensor to have size at least 61 at dimension 1, but got size 32 for argument #2 'targets' (while checking arguments for ctc_loss_gpu)

[felixdittrich92]

Looks like there is an sample in your dataset with more than 32 characters (default max_length is 32 for the crnn models) :)

[ep0p]

Thanks a lot @felixdittrich92 , finally it works.