Cross Mark Detection in Doctr

[jidalii]

Hi, I currently use docTR to convert scanned PDF files into CSV files. One section of the sheet contains cross marks. However, I tried different text detection models, and neither can detect cross marks. I wonder whether there are any missing features from docTR that I overlooked for the cross mark detection, or there is an alternative solution for it. Thanks!

Here is the image and screenshot of the detection result:

Here is the code:

predictor = ocr_predictor(
    det_arch="fast_base",
    reco_arch="crnn_vgg16_bn",
    pretrained=True,
    assume_straight_pages=True,
    detect_orientation=True,
    straighten_pages=False,
)
doc = DocumentFile.from_images(filename)
result = predictor(doc)
result.show()

[felixdittrich92]

Hi @jidalii 👋,

One thing you could try is to binarize the image because I think the models have maybe problems with the blue X.

doc = DocumentFile.from_images(image_paths)
# Binaraize - adjust bin scores to your needs
doc = [
    cv2.merge([
        cv2.threshold(cv2.cvtColor(page, cv2.COLOR_BGR2GRAY), 220, 255, cv2.THRESH_BINARY)[1]
    ] * 3)
    for page in doc
]

Another option would be to fine tune the preferred model (for this case a few samples should be enough ~20-40)

See: https://mindee.github.io/doctr/using_doctr/custom_models_training.html

A labeling tool can be found here: https://github.com/text2knowledge/docTR-Labeler - early phase release (Or take any other tool of your choice) - boxes should be as close as possible the the text (word)

Best,
Felix