Description
I try to train/eval/export mobilenetv3_large_cityscapes_trainfine and mobilenetv3_small_cityscapes_trainfine which mentioned in https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md.
However, I could not run my code successfully. For example, I try to export mobilenetv3_large_cityscapes_trainfine ckeckpoint to frozen_inference_graph.pb by adding the additional commandline flags in the model_zoo.md, the full script is
python "${WORK_DIR}"/export_model.py
--logtostderr
--checkpoint_path="${CKPT_PATH}"
--export_path="${EXPORT_PATH}"
--model_variant="mobilenet_v3_large_seg"
--num_classes=19
--crop_size=513
--crop_size=513
--inference_scales=1.0
--image_pooling_crop_size=769,769
--image_pooling_stride=4,5
--add_image_level_feature=1
--aspp_convs_filters=128
--aspp_with_concat_projection=0
--aspp_with_squeeze_and_excitation=1
--decoder_use_sum_merge=1
--decoder_filters=19
--decoder_output_is_logits=1
--image_se_uses_qsigmoid=1
--decoder_output_stride=8
--output_stride=32
which WORK_DIR, CKPT_PATH and EXPORT_PATH are correctly input and encounter the error log as follows:
Please use layer.__call__ method instead.
Traceback (most recent call last):
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/export_model.py", line 201, in
tf.app.run()
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/export_model.py", line 134, in main
image_pyramid=FLAGS.image_pyramid)
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/model.py", line 188, in predict_labels
fine_tune_batch_norm=False)
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/model.py", line 323, in multi_scale_logits
nas_training_hyper_parameters=nas_training_hyper_parameters)
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/model.py", line 577, in _get_logits
nas_training_hyper_parameters=nas_training_hyper_parameters)
File "/group/modelzoo/internal-cooperation-models/IPU/tf_deeplabv3/code/research/deeplab/model.py", line 467, in extract_features
model_options.image_pooling_stride, padding='VALID')
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py", line 122, in avg_pool2d
outputs = layer.apply(inputs)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 1700, in apply
return self.call(inputs, *args, **kwargs)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/layers/base.py", line 548, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 854, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 237, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in converted code:
relative to /group/dphi_algo_scratch_13/fangyuan/anaconda3/envs/tf1_test/lib/python3.7/site-packages/tensorflow_core/python:
keras/layers/pooling.py:248 call
data_format=conv_utils.convert_data_format(self.data_format, 4))
ops/nn_ops.py:3583 avg_pool
name=name)
ops/gen_nn_ops.py:95 avg_pool
data_format=data_format, name=name)
framework/op_def_library.py:794 _apply_op_helper
op_def=op_def)
util/deprecation.py:507 new_func
return func(*args, **kwargs)
framework/ops.py:3357 create_op
attrs, op_def, compute_device)
framework/ops.py:3426 _create_op_internal
op_def=op_def)
framework/ops.py:1770 init
control_input_ops)
framework/ops.py:1610 _create_c_op
raise ValueError(str(e))
ValueError: Negative dimension size caused by subtracting 25 from 17 for 'AvgPool2D/AvgPool' (op: 'AvgPool') with input shapes: [1,17,17,480].
Moreover, I noticed that maybe I should changed the image size to adapt to cityscape dataset with following scripts:
python "${WORK_DIR}"/export_model.py
--logtostderr
--checkpoint_path="${CKPT_PATH}"
--export_path="${EXPORT_PATH}"
--model_variant="mobilenet_v3_large_seg"
--num_classes=19
--crop_size=1024
--crop_size=2048
--inference_scales=1.0
--image_pooling_crop_size=769,769
--image_pooling_stride=4,5
--add_image_level_feature=1
--aspp_convs_filters=128
--aspp_with_concat_projection=0
--aspp_with_squeeze_and_excitation=1
--decoder_use_sum_merge=1
--decoder_filters=19
--decoder_output_is_logits=1
--image_se_uses_qsigmoid=1
--decoder_output_stride=8
--output_stride=32
which change the image size to [1024, 2048] which is image size in cityscape, however, the export pb model has lower mIoU than the released results(from 72.41->61.04)
so what is the correct script for mobilenetv3 in deeplab codebase?