Unable to deploy ssd-fpn model to mobile

[AliceDinh]

System information

• What is the top-level directory of the model you are using: object_detection
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Window 10
• TensorFlow installed from (source or binary): binary, installed from pip: pip install tensorflow
• TensorFlow version (use command below): 1.12.0
• Bazel version (if compiling from source): None
• CUDA/cuDNN version: None
• GPU model and memory: None
• Exact command to reproduce: None

Describe the problem

I am not able to deploy ssd-fpn model to mobile.
I used TensorFlow Android Camera Demo, which is TF Detect, the ssd-fpn model is download from model zoo but nothing detected.

[maxcrous]

Thanks for creating a separate issue on this.
@achowdhery and I went over the details of reproducing the error in #5298.
They suspect it is a bug with Mul op, and will be looking into it in the next few days.

[oopsodd]

I used TF object detection API to train ssd_resnet_50_fpn_coco with a 50-classes dataset.
Everything is okay with frozen model.
The checkpoint was converted successfully.
But the tflite model detect wrong class, bbox. All the output classes are the same (1 class).
The tflite model takes 3s per image to inference on Galaxy S9 (same as the frozen model).
Did Tflite support ssd_resnet_50_fpn_coco?

@AliceDinh Ban su dung mobilenet hay resnet?

[AliceDinh]

@oopsodd I used mobilenet but could not deploy to mobile, so you deploy resnet fpn to mobile successfully, right?
So I have to convert my ssd-mobilenet-fpn frozen model to tflite model before deploy?

[oopsodd]

@AliceDinh Ban lam theo huong dan o day dung k, voi inference type la Float?
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md
Freeze model truoc bang export_tflite_ssd_graph.py
Co ve nhu tflite van chua ho tro FPN dau :(

[AliceDinh]

@oopsodd To train model theo huong dan nay co: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_locally.md
Sau do thi export_inference_graph su dung script o link nay: https://github.com/tensorflow/models/blob/master/research/object_detection/export_inference_graph.py, thu duoc frozen_inference_graph.pb
Cuoi cung, thi import frozen_inference_graph.pb vao app nay: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android thi khong nhan dang duoc.
P/s: frozen_inference_graph.pb của SSD-Mobilenet thi van duoc, nhưng ssd-fpn-mobilenet thi khong duoc

[AliceDinh]

The checkpoint was converted successfully.

@oopsodd You converted by using object_detection/export_tflite_ssd_graph.py or using tflite_convert of toco?

[oopsodd]

Ban dau minh tuong ban noi den tflite, minh convert sang frozen bang export-ssd-grah, sau do dung tiep TOCO. Theo dung huong dan convert sang tflite.

[oopsodd]

Ve frozen model export bang export-inference-graph, ben minh co nguoi deploy va report la la ssd resnet fpn detect duoc. Minh k truc tiep sua android camera demo app nen khong ro : (

[AliceDinh]

@oopsodd Yep, ban dau minh convert sang frozen bang: object_detection/export_inference_graph.py, sau đó dùng TOCO tflite_convert thì đang có lỗi tflite_convert - ValueError: Invalid tensors 'input' were found. Cho minh hoi trong khoi lenh convert thì --input_arrays và --output_arrays là gì?

[oopsodd]

Trong nay huong dan day du roi ma.
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md
Input, output giong example command trong ay luon.

[thusinh1969]

Làm như này:

Convert into tflite-compatible pb. Lưu ý --max_detections:

python export_tflite_ssd_graph.py --input_type image_tensor --pipeline_config_path training_ssd_v2_config --trained_checkpoint_prefix ../output/model.ckpt-124960 --output_directory ../output/frozen_tflite/frozen_inference_graph.pb -add_postprocessing_op True --max_detections 10

--> Sinh ra 2 file tflite_graph.pb và tflite_graph.pbtxt

Sau đó make sure là chạy Tensorflow (hay Tensorflow GPU) bản 1.11 trở lên:

tflite_convert --output_file=./tflite/detect.tflite
--graph_def_file=/home/nguyen/ssd/Foods/dataset/OD_food_non_food/ssd_model/output/frozen_tflite/frozen_inference_graph.pb/tflite_graph.pb
--input_arrays=normalized_input_image_tensor
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'
--input_shape=1,300,300,3
--allow_custom_ops

**Ăn ngay 🍡 **