How to leverage multiprocessing for inference?

[devendrapal5755]

How to use multiprocessing into inference ?

[fg-mindee]

Hi @devendrapal5755 👋

Could you elaborate on your question please?
Do you mean performing inference on several pages at the same time for OCR? Do you mean batches of samples for sub-tasks (text detection, text recognition)?

If you meant any of the previous two, our predictor objects will do the job for you!

from doctr.io import DocumentFile
from doctr.models import ocr_predictor

doc = DocumentFile.from_images(["path/to/page1.jpg", "path/to/page2.jpg"])
# The predictor will handle the multiprocessing on python-level, and batching on deep-learning ops
model = ocr_predictor(pretrained=True)
result = model(doc)

Let me know if I misunderstood your question!

[devendrapal5755]

performing inference on several pages at the same time for OCR its use multiprocessing technique or not , if yes so please explain me.

[fg-mindee]

? well, as I just said, it is supported and done by default with the script above 😅

Under-the-hood:

• each predictor chunk the input in batches
• it preprocesses those using multi-threading
• the model performs inference on the batch

If that doesn't answer your question, would you mind being more specific about what you're looking for please? 🙏

[devendrapal5755]

Thank you so much, I got it.