Fontconfig error during training recognition

[CukoF]

Hello everyone,

First of all, thank you very much for this beautiful software and your efforts. It is exciting to work with your OCR.

I'm new to image processing. I don't have much knowledge, so my question may be very simple or silly. I apologise for this.

I have prepared the necessary datasets containing Turkish texts for Text Recognition. I divided them into train and validation. I also added Turkish VOCABS. However, when I run the command for train, I get the following output.

python references/recognition/train_pytorch.py crnn_vgg16_bn --train_path "train/dataset/path" --val_path "val/dataset/path" --epochs 5 --device 0 --vocab turkish

Output:

Fontconfig error: Cannot load default config file: No such file: (null)
Fontconfig error: Cannot load default config file: No such file: (null)
Fontconfig error: Cannot load default config file: No such file: (null)
Fontconfig error: Cannot load default config file: No such file: (null)

What is the reason for this? Although I set the fontconfig path, this problem still persists.Epochs are progressing but I suspect something is wrong.

[felixT2K]

Hi @CukoF 👋🏼,

Great to see you like docTR ❤️

Could you check it please with --show-samples

[felixT2K]

Sounds good 👍🏼

[CukoF]

When I use the command like this ;
python references/recognition/train_pytorch.py crnn_vgg16_bn --train_path "train/dataset/path" --val_path "val/dataset/path" --epochs 5 --device 0 --vocab turkish --show-samples

I got that output;

Fontconfig error: Cannot load default config file: No such file: (null)
Namespace(arch='crnn_vgg16_bn', train_path='C:\\Users\traindataset\path', val_path='C:\\Users\val\dataset\path', train_samples=1000, val_samples=20, font='FreeMono.ttf,FreeSans.ttf,FreeSerif.ttf', min_chars=1, max_chars=12, name=None, epochs=1, batch_size=64, device=0, input_size=32, lr=0.001, weight_decay=0, workers=None, resume=None, vocab='turkish', test_only=False, show_samples=True, wb=False, push_to_hub=False, pretrained=False, sched='cosine', amp=False, find_lr=False)
Validation set loaded in 0.9998s (58259 samples in 911 batches)
Train set loaded in 5.856s (232741 samples in 3636 batches)
Fontconfig error: Cannot load default config file: No such file: (null)

And also I have one more issue when I try to load my model.

import torch
from doctr.models import ocr_predictor, crnn_vgg16_bn
from doctr.datasets import VOCABS

reco_model = crnn_vgg16_bn(pretrained=False, pretrained_backbone=False, vocab=VOCABS["turkish"])
reco_params = torch.load('./doctr-reco-train/crnn_vgg16_bn_20240524-132321.pt', map_location="cpu")
reco_model.load_state_dict(reco_params)

predictor = ocr_predictor(det_arch='linknet_resnet18', reco_arch=reco_model, pretrained=True)

When I run that code, I got RuntimeError like below;

RuntimeError                              Traceback (most recent call last)
Cell In[9], [line 7](vscode-notebook-cell:?execution_count=9&line=7)
      [5](vscode-notebook-cell:?execution_count=9&line=5) reco_model = crnn_vgg16_bn(pretrained=False, pretrained_backbone=False, vocab=VOCABS["turkish"])
      [6](vscode-notebook-cell:?execution_count=9&line=6) reco_params = torch.load('./doctr-reco-train/crnn_vgg16_bn_20240524-132321.pt', map_location="cpu")
----> [7](vscode-notebook-cell:?execution_count=9&line=7) reco_model.load_state_dict(reco_params)
      [9](vscode-notebook-cell:?execution_count=9&line=9) predictor = ocr_predictor(det_arch='linknet_resnet18', reco_arch=reco_model, pretrained=True)

File c:\Users\/..../\doctr_env\Lib\site-packages\torch\nn\modules\module.py:2189, in Module.load_state_dict(self, state_dict, strict, assign)
   [2184](file:///C:/Users/....//doctr_env/Lib/site-packages/torch/nn/modules/module.py:2184)         error_msgs.insert(
   [2185](file:///C:/Users//....//doctr_env/Lib/site-packages/torch/nn/modules/module.py:2185)             0, 'Missing key(s) in state_dict: {}. '.format(
   [2186](file:///C:/Users//....//doctr_env/Lib/site-packages/torch/nn/modules/module.py:2186)                 ', '.join(f'"{k}"' for k in missing_keys)))
   [2188](file:///C:/Users//..../doctr_env/Lib/site-packages/torch/nn/modules/module.py:2188) if len(error_msgs) > 0:
-> [2189](file:///C:/Users/..../doctr_env/Lib/site-packages/torch/nn/modules/module.py:2189)     raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   [2190](file:///C:/Users//....//doctr_env/Lib/site-packages/torch/nn/modules/module.py:2190)                        self.__class__.__name__, "\n\t".join(error_msgs)))
   [2191](file:///C:/Users/..../doctr_env/Lib/site-packages/torch/nn/modules/module.py:2191) return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for CRNN:
	size mismatch for linear.weight: copying a param with shape torch.Size([174, 256]) from checkpoint, the shape in current model is torch.Size([140, 256]).
	size mismatch for linear.bias: copying a param with shape torch.Size([174]) from checkpoint, the shape in current model is torch.Size([140]).

[felixT2K]

Ah yeah sure i see we don't have a turkish vocab predefined yet see:

https://github.com/mindee/doctr/blob/main/doctr/datasets/vocabs.py

Two options:

1. You open a PR and add it in https://github.com/mindee/doctr/blob/main/doctr/datasets/vocabs.py + https://github.com/mindee/doctr/blob/main/docs/source/modules/datasets.rst

2. you install the cloned repo with pip install -e . (editable mode) and add it your own to https://github.com/mindee/doctr/blob/main/doctr/datasets/vocabs.py

PS: I think you did the sec option already otherwise it should raise an KeyError 😅

[felixT2K]

The code looks correct so could you check your added vocab again please because if the training works the loading should also (does nothing different to init the model)

[CukoF]

Problem solved. I was able to install the model I trained and tried it on a test case.

As you said, I rechecked the Turkish vocabs I added afterwards and removed unnecessary characters. I also synchronised the doctr files in .venv and the vocab.py files of the repo I'm currently in. It's working fine now.

Thank you very much for your quick answers.

I can't wait to see the results after training with more Turkish data and texts. ❤️