Using doctr with layoutLMv3

[viraj071]

I am trying to build a system to classify documents in (PDF, Image) formats. I am trying to use LayoutLMv3 for the classification task and doctr for the OCR. Does anyone know how I can pass the words and bounding boxes into the LayoutLMv3 processor? Any help or leads is appreciated.

[felixdittrich92]

Hi @viraj071 👋 ,
take a look at the code snippet i have provided in #1088
In your case no need to rescale the coords you can pass it as is (processor needs normalized coords)
So easily save words and coords in a list and pass it into the processor :)

[viraj071]

Thank you. When you multiply the values by the width and the height, the values can go over 1000 but LayoutLMv3 needs them to be less than 1000.

[felixdittrich92]

Yeah no need to do this as explained coords are already normalized :)

[viraj071]

I am currently doing this
[geometry[0][0]/width * 1000, geometry[0][1] / height * 1000, geometry[1][0] / width * 1000, geometry[1][1] / height * 1000].

Is the above required or are the normalized coords ready to use directly?

[felixdittrich92]

Hi @viraj071,

I think you have to do the following:
take #1088 (coords corresponding to original image size) code snippet as is and do the following:

box = [
  int(1000 * (xmin / width)),
  int(1000 * (ymin / height)),
  int(1000 * (xmax / width)),
  int(1000 * (ymax / height)),
]