Boundary box created using absolute coordinates is not in the correct position.

[NalinMalla]

I have been trying to draw bbox on a picture of the document which I am trying to ocr to see the detected lines of text. The problem I am facing is the absolute coordinates of the bboxes I calculated, by multiplying the relative coords and page dimensions, when drawn on the original image are all shifted towards the top right corner. Is there a way to correct this?

This is my code for calculating bboxes:

from doctr.models import ocr_predictor
from doctr.io import DocumentFile

# Use docTR to analyze the image and obtain the result
line_boundaries = []
model = ocr_predictor(pretrained=True)  #setting preserve_aspect_ratio=False or symmetric_pad=False didn't make a difference.
doc = DocumentFile.from_images(img_path)
result = model(doc)

# Extract bounding box coordinates for each line
for page in result.pages:
    for block in page.blocks:
        for line in block.lines:
          #Multiplied relative coordinates with page dimension to get absolute coordinates
            x_min, y_min, x_max, y_max = round(line.geometry[0][0] * page.dimensions[0]), round(line.geometry[0][1] * page.dimensions[1]), round(line.geometry[1][0] * page.dimensions[0]), round(line.geometry[1][1] * page.dimensions[1])
            line_boundaries.append((x_min, y_min, x_max, y_max))

This is the value of line_boundaries:
[(531, 148, 1321, 184), (2725, 148, 3061, 177), (526, 254, 3071, 295), (526, 288, 3071, 332), (535, 324, 3071, 363), ... ]

This is the function I used to draw the boxes:

import cv2
from google.colab.patches import cv2_imshow #to use instead of cv2.imshow as it causes collab to crash

def draw_rectangles(image_path, line_boundaries):
  """
  Draws rectangles on an image using the provided line boundaries.

  Args:
    image_path: The path to the image file.
    line_boundaries: A list of line boundaries, where each boundary is a list of four points.

  Returns:
    None
  """

  # Load the image
  image = cv2.imread(img_path)

  # Iterate over the line boundaries and draw rectangles
  for boundary in line_boundaries:
    #x1, y1, x2, y2 = int(boundary[0][0]), int(boundary[0][1]), int(boundary[2][0]), int(boundary[2][1])
    x1, y1, x2, y2 = map(int, boundary)  # Convert coordinates to integers
    cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)

  # Display the image with rectangles
  cv2_imshow(image)  # Use cv2_imshow instead of cv2.imshow for collab only
  cv2.waitKey(0)
  cv2.destroyAllWindows()

This is the image with bboxes drawn on it.

I have tried it without the round-off which made no difference and have also tried using the predictor only but to no avail. Setting preserve_aspect_ratio=False or symmetric_pad=False in the ocr_predictor also didn't make a difference.

[hienphan161]

Hi. This one may help #570

[NalinMalla]

Hi @hienphan161, I am aware of the thread you referred and have followed its instructions but it doesn't solve the problem. Still I appreciate you response.

[hienphan161]

Following that thread, I think you need to swap page.dimensions[0] and page.dimensions[1] when calculating the absolute coordinates. Have you tried that?

[NalinMalla]

Wow that's embarrassing, I don't know how I didn't catch that. A fresh pair of eyes really helped, thanks for pointing out my mistake @hienphan161, it fixed the problem.