performances issues on finetuning db_resnet50 on cord

[nikokks]

Hi all,

When I try to finetune a db_resnet50 on CORD I get 83.44 % of f1-score
But when we see that paper https://arxiv.org/pdf/2210.07903 we should get

Why does doctr have no the same performances ?

[felixdittrich92]

Hey @nikokks 🤗

That's a really hard question because of so many factors like: post-pre -processing / used augmentations / training configuration / etc.

Maybe a short question have you used the new augmentation pipeline (#1577) or the old one ?

[nikokks]

I actually used the new script with the new data augmentation that you pushed (congratulations again!)
I obtained these performances on CORD with the db_resnet50, with the rotation True and with eval_straight False

For my part, I was able to investigate and it turns out that doctr uses retacular polygons whereas for most open source detection models they use polygons with more than 4 sides which boosts their detection performance.

here for example are text masks without rectangles from different opensource models
https://github.com/GXYM/TextBPN
or
https://github.com/WenmuZhou/DBNet.pytorch

it turns out that in this paper (already mentioned previously https://arxiv.org/pdf/2210.07903) non-rectangular detection with more than 4 sides does not decreased the text recognition performances

do you think that doctr could allow polygons with more than 4 sides (non-rectangular?)

what do you think about it ? =)