Using MobileNetV3Backbone for object detection

[barrypitman]

Hi,

Firstly, thanks for a great project, its been exciting to follow the progress.

I'm interested in doing object detection on edge devices in latency-critical applications. In the past, I've had success with ssd+mobilenetv2 with tensorflow models from the model zoo.

I see that you have a MobileNetV3Backbone with weights for 'mobilenet_v3_large_imagenet'. However, I'm not sure how I'd use that backbone in an object detection task. Is there currently any way to do that using an existing model from the object_detection/ directory? Or are the backbones primarily for use in classification tasks?

Thanks in advance!

[jbischof]

Thanks for the question @barrypitman!

Our object detection models Task models are compatible with any of our Backbone models. While they are pretrained on Imagenet, this is the classic source of transfer learning for all CV tasks.

If you want to use our mobilenet preset with RetinaNet, you can use one of two APIs:

# Explicit backbone
backbone = keras_cv.models.MobileNetV3Backbone.from_preset("mobilenet_v3_large_imagenet")
model = keras_cv.models.RetinaNet(
    backbone,
    num_classes=20,
    bounding_box_format="xywh",
)
# from_preset constructor
model = keras_cv.models.RetinaNet.from_preset(
    "mobilenet_v3_large_imagenet",
    num_classes=20,
    bounding_box_format="xywh",
)

[jbischof]

Also make sure to check out our object detection guide: https://keras.io/guides/keras_cv/object_detection_keras_cv/

[barrypitman]

I've run into some issues using the MobileNetV3Backbone with RetinaNet. When I try run an image through your example model, I get an error:

backbone = keras_cv.models.MobileNetV3Backbone.from_preset("mobilenet_v3_large_imagenet")
model = keras_cv.models.RetinaNet(
    backbone,
    num_classes=20,
    bounding_box_format="xywh",
)

img = tf.ones(shape=(1, 512, 512, 3))
output = model.predict(img, verbose=0)

in user code:
    
        File "C:\Users\barry\work\keras_object_detection\src\main\python\venv_idea\lib\site-packages\keras_cv\models\object_detection\retinanet\feature_pyramid.py", line 46, in call  *
            p3_output = p3_output + self.upsample_2x(p4_output, training=training)
    
        ValueError: Dimensions must be equal, but are 64 and 128 for '{{node retina_net/feature_pyramid/add_1}} = AddV2[T=DT_FLOAT](retina_net/feature_pyramid/conv2d_18/BiasAdd, retina_net/feature_pyramid/up_sampling2d/resize_1/ResizeNearestNeighbor)' with input shapes: [?,64,64,256], [?,128,128,256].
    
    
    Call arguments received by layer 'feature_pyramid' (type FeaturePyramid):
      • inputs={3: 'tf.Tensor(shape=(None, 64, 64, 24), dtype=float32)', 4: 'tf.Tensor(shape=(None, 64, 64, 24), dtype=float32)', 5: 'tf.Tensor(shape=(None, 32, 32, 40), dtype=float32)'}
      • training=False

I've tried with various input sizes, e.g. 128, 256, 300, 320, 512, 640, and I get the same result.

If I try the same with the resnet50_imagenet backbone, it works.

[jbischof]

Thanks @barrypitman, could you file this as an issue on our repo? We'll triage this ASAP