Low FPS in YoloV4 - OpenVINO model

[vbarunn]

Hi, I have trained the YOLOV4 custom model with 3 classes. In order to improve the video inference timing (High FPS), I have optimized it with OpenVINO. However, the inference timing is not improved (got around 3FPS). Any suggestion to improve the timing?.
The command used for inference:
python object_detection_demo_yolov3_async.py -i video.mp4 -m frozen_darknet_yolov4_model_weapon.xml -d CPU

For model optimization:
python "C:\Program Files (x86)\Intel\openvino_2021.3.394\deployment_tools\model_optimizer\mo.py" --input_model frozen_darknet_yolov4_model.pb --transformations_config yolov4.json --batch 1 --reverse_input_channels

yolov4.josn:
[
{
"id": "TFYOLOV3",
"match_kind": "general",
"custom_attributes": {
"classes": 3,
"anchors": [12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401],
"coords": 4,
"num": 9,
"masks":[[0, 1, 2], [3, 4, 5], [6, 7, 8]],
"entry_points": ["detector/yolo-v4/Reshape", "detector/yolo-v4/Reshape_4", "detector/yolo-v4/Reshape_8"]
}
}
]

[TNTWEN]

@vbarunn
Reducing classes will not make the model faster.So it's normal that your model(3 classes) only have 3FPS in OpenVINO.
I have provided a solution to speed up the embedded devices which was listed in development log：Model pruning +INT8 Quantization

Pruned-OpenVINO-YOLO：https://github.com/TNTWEN/Pruned-OpenVINO-YOLO
INT8 Quantization：https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/master/INT8

Many users try this solution successfully，they prune their custom yolov4 model（1 class, weights 245M )to 800Kb,Speed is increased from 3fps to 30fps with little loss of accuracy in OpenVINO(cpu)

But Pruned-OpenVINO-YOLO needs sufficient GPU resources and time