YoloV4 broken with OpenVINO 2021.X

[sandeepmohanadasan]

Hi @TNTWEN , I am getting an error while converting YoloV4 weight to OPENVINO format. I have tried the 2021.1 and 2021.3 version. I can replicate the issue on both versions.
It will be a great help if you can guide me on this.

(env) C:\Users\<user>\conversion\OpenVINOModelConversion>python convert_weights_pb.py --class_names cfg/coco_chip.names --weights_file ChipstackTrainedModel/custom-yolov4-detector_last.weights --data_format NHWC


WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From convert_weights_pb.py:52: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0526 19:08:50.889183 16784 module_wrapper.py:139] From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0526 19:08:51.322626 16784 module_wrapper.py:139] From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\.conda\envs\openvino\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W0526 19:08:51.344624 16784 deprecation.py:323] From C:\Users\sandeep.mohanadasan.SG\.conda\envs\openvino\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\yolo_v4.py:111: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

W0526 19:08:53.571855 16784 module_wrapper.py:139] From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\yolo_v4.py:111: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

WARNING:tensorflow:From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

W0526 19:08:54.875165 16784 module_wrapper.py:139] From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

W0526 19:08:56.247044 16784 module_wrapper.py:139] From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W0526 19:08:57.728939 16784 module_wrapper.py:139] From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-05-26 19:08:57.806688: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W0526 19:09:07.070686 16784 module_wrapper.py:139] From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
W0526 19:09:08.241832 16784 deprecation.py:323] From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\.conda\envs\openvino\lib\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
W0526 19:09:08.251964 16784 deprecation.py:323] From C:\Users\sandeep.mohanadasan.SG\.conda\envs\openvino\lib\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
INFO:tensorflow:Froze 541 variables.
I0526 19:09:09.357801 16784 graph_util_impl.py:334] Froze 541 variables.
INFO:tensorflow:Converted 541 variables to const ops.
I0526 19:09:09.744303 16784 graph_util_impl.py:394] Converted 541 variables to const ops.
WARNING:tensorflow:From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W0526 19:09:09.807855 16784 module_wrapper.py:139] From C:\Users\sandeep.mohanadasan.SG\conversion\OpenVINOModelConversion\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

1808 ops written to frozen_darknet_yolov4_model.pb.

Intel Driver details

Packages List

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
_tflow_select=2.2.0=eigen
absl-py=0.12.0=py36haa95532_0
aiohttp=3.7.4=py36h2bbff1b_1
astor=0.8.1=py36haa95532_0
astunparse=1.6.3=pypi_0
async-timeout=3.0.1=py36haa95532_0
attrs=21.2.0=pyhd3eb1b0_0
blas=1.0=mkl
bleach=1.5.0=pypi_0
blinker=1.4=py36haa95532_0
brotlipy=0.7.0=py36h2bbff1b_1003
ca-certificates=2020.12.5=h5b45459_0
cached-property=1.5.2=pypi_0
cachetools=4.2.2=pyhd3eb1b0_0
certifi=2020.12.5=py36ha15d459_1
cffi=1.14.5=py36hcd4344a_0
chardet=3.0.4=py36haa95532_1003
click=8.0.1=pyhd3eb1b0_0
cmake=3.13.0=pypi_0
coverage=5.5=py36h2bbff1b_2
cryptography=3.4.7=py36h71e12ea_0
cython=0.29.23=py36hd77b12b_0
flatbuffers=1.12=pypi_0
gast=0.2.2=pypi_0
google-auth=1.30.0=pyhd3eb1b0_0
google-auth-oauthlib=0.4.4=pyhd3eb1b0_0
google-pasta=0.2.0=py_0
grpcio=1.34.1=pypi_0
h5py=2.10.0=py36h5e291fa_0
hdf5=1.10.4=h7ebc959_0
html5lib=0.9999999=pypi_0
icc_rt=2019.0.0=h0cc432a_1
idna=2.10=pyhd3eb1b0_0
idna_ssl=1.1.0=py36haa95532_0
importlib-metadata=3.10.0=py36haa95532_0
intel-openmp=2021.2.0=haa95532_616
keras-applications=1.0.8=py_1
keras-nightly=2.5.0.dev2021032900=pypi_0
keras-preprocessing=1.1.2=pyhd3eb1b0_0
libprotobuf=3.14.0=h23ce68f_0
markdown=3.3.4=py36haa95532_0
mkl=2020.2=256
mkl-service=2.3.0=py36h196d8e1_0
mkl_fft=1.3.0=py36h46781fe_0
mkl_random=1.1.1=py36h47e9c7a_0
multidict=5.1.0=py36h2bbff1b_2
numpy=1.18.5=pypi_0
oauthlib=3.1.0=py_0
openssl=1.1.1k=h8ffe710_0
opt_einsum=3.3.0=pyhd3eb1b0_1
pillow=8.2.0=pypi_0
pip=21.1.1=py36haa95532_0
protobuf=3.14.0=py36hd77b12b_1
pyasn1=0.4.8=py_0
pyasn1-modules=0.2.8=py_0
pycparser=2.20=py_2
pyjwt=1.7.1=py36_0
pyopenssl=20.0.1=pyhd3eb1b0_1
pyreadline=2.1=py36_1
pysocks=1.7.1=py36haa95532_0
python=3.6.5=h0c2934d_0
python_abi=3.6=1_cp36m
requests=2.25.1=pyhd3eb1b0_0
requests-oauthlib=1.3.0=py_0
rsa=4.7.2=pyhd3eb1b0_1
scipy=1.5.2=py36h9439919_0
setuptools=52.0.0=py36haa95532_0
six=1.15.0=py36haa95532_0
tensorboard=1.15.0=pypi_0
tensorboard-data-server=0.6.1=pypi_0
tensorboard-plugin-wit=1.6.0=py_0
tensorflow=1.15.5=pypi_0
tensorflow-estimator=1.15.1=pyh2649769_0
tensorflow-tensorboard=1.5.1=pypi_0
termcolor=1.1.0=py36haa95532_1
typing-extensions=3.7.4.3=hd3eb1b0_0
typing_extensions=3.7.4.3=pyh06a4308_0
urllib3=1.26.4=pyhd3eb1b0_0
vc=14.2=h21ff451_1
vs2015_runtime=14.27.29016=h5e58377_2
werkzeug=0.16.1=py_0
wheel=0.36.2=pyhd3eb1b0_0
win_inet_pton=1.1.0=py36haa95532_0
wincertstore=0.2=py36h7fe50ca_0
wrapt=1.12.1=py36he774522_1
yarl=1.6.3=py36h2bbff1b_0
zipp=3.4.1=pyhd3eb1b0_0
zlib=1.2.11=h62dcd97_4

[TNTWEN]

@sandeepmohanadasan
what error you met？
1808 ops written to frozen_darknet_yolov4_model.pb. It runs correctly . No error
It's just the first strp,the tensorflow program .
mo.py is OpenVINO program

[sandeepmohanadasan]

So can I ignore the above-mentioned warnings as the .pb file generated?

[TNTWEN]

@sandeepmohanadasan
Yes! Just warnings,not errors

[sandeepmohanadasan]

@TNTWEN
Thanks for the clarification. I have proceeded with the remaining commands .Let me know if you can throw some light on below issue

- call C:\Program Files (x86)\Intel\openvino_2021\bin\setupvars.bat

- python convert_weights_pb.py --weights_file Model\custom-yolov4-detector_last.weights --output_graph model_yolov4.pb --class_names cfg\coco_chip.names --data_format NHWC --size 416

- python C:\Program Files (x86)\Intel\openvino_2021.1.110\deployment_tools\model_optimizer\mo.py --input_model model_yolov4.pb --transformations_config model_yolov4.json --reverse_input_channels --batch 1

below files generated

- model_yolov4.bin
- model_yolov4.mapping
- model_yolov4.xml
- model_yolov4.pb

But when I tried to validate my image, I was getting the below error

(env) C:\Users\<user>\conversion\OpenVINOModelConversion>python object_detection_demo_yolov4_async2021.py -m model_yolov4.xml -i modeltest01.jpg -d CPU
[ INFO ] Creating Inference Engine...
[ INFO ] Loading network
[ INFO ] Preparing inputs
MFX: Unsupported extension: modeltest01.jpg
[ INFO ] Loading model to the plugin
[ INFO ] Starting inference...
To close the application, press 'CTRL+C' here or switch to the output window and press ESC key
To switch between min_latency/user_specified modes, press TAB key in the output window
object_detection_demo_yolov4_async2021.py:233: DeprecationWarning: 'layers' property of IENetwork class is deprecated. For iteration over network please use get_ops()/get_ordered_ops() methods from nGraph Python API
  out_blob = out_blob.buffer.reshape(net.layers[net.layers[layer_name].parents[0]].out_data[0].shape)
Traceback (most recent call last):
  File "object_detection_demo_yolov4_async2021.py", line 496, in <module>
    sys.exit(main() or 0)
  File "object_detection_demo_yolov4_async2021.py", line 386, in main
    args.keep_aspect_ratio)
  File "object_detection_demo_yolov4_async2021.py", line 233, in get_objects
    out_blob = out_blob.buffer.reshape(net.layers[net.layers[layer_name].parents[0]].out_data[0].shape)
ValueError: cannot reshape array of size 105456 into shape (1,33,52,52)

[TNTWEN]

It is obvious that you didn't set correct classes in model_yolov4.json
105456 is (1,39,52,52) means classes =8
(1,33,52,52) means classes=5

check your classes file and corresponding parameters