Skip to content

Latest commit

 

History

History
191 lines (124 loc) · 6.57 KB

README.md

File metadata and controls

191 lines (124 loc) · 6.57 KB
Raw

yolov8_seg

Table of contents

1. Description

The model used in this example comes from the following open source projects:

https://github.com/airockchip/ultralytics_yolov8

2. Current Support Platform

RK3566, RK3568, RK3588, RK3562, RK1808, RV1109, RV1126

3. Pretrained Model

Download link:

yolov8n-seg.onnx
yolov8s-seg.onnx
yolov8m-seg.onnx

Download with shell command:

cd model
./download_model.sh

Note: The model provided here is an optimized model, which is different from the official original model. Take yolov8n-seg.onnx as an example to show the difference between them.

  1. The comparison of their output information is as follows. The left is the official original model, and the right is the optimized model. As shown in the figure, the original output [1,116,8400] is divided into three groups. For example, in the set of outputs ([1,64,80,80],[1,80,80,80],[1,1,80,80],[1,32,80,80]),[1,64,80,80] is the coordinate of the box, [1,80,80,80] is the confidence of the box corresponding to the 80 categories, [1,1,80,80] is the sum of the confidence of the 80 categories, and [1,32,80,80] is the segmentation feature.
Image
  1. Taking the the set of outputs ([1,64,20,20],[1,80,20,20],[1,1,20,20],[1,32,20,20]) as an example, we remove the subgraphs behind the three convolution nodes in the model (the framed part in the figure), keep the outputs of these three convolutions ([1,64,20,20],[1,80,20,20],[1,32,20,20]), and add a reducesum+clip branch for calculating the sum of the confidence of the 80 categories ([1,1,20,20]).
Image

4. Convert to RKNN

Usage:

cd python
python convert.py <onnx_model> <TARGET_PLATFORM>
# such as: python convert.py ../model/yolov8s-seg.onnx rk3566
# output model will be saved as ../model/yolov8_seg.rknn

Description:

  • <onnx_model>: Specify ONNX model path.
  • <TARGET_PLATFORM>: Specify NPU platform name. Support Platform refer [here](#2 Current Support Platform).
  • <dtype>(optional): Specify as i8, u8 or fp. i8/u8 for doing quantization, fp for no quantization. Default is i8/u8.
  • <output_rknn_path>(optional): Specify save path for the RKNN model, default save in the same directory as ONNX model with name yolov8_seg.rknn

5. Python Demo

Usage:

cd python
# Inference with ONNX model
python yolov8_seg.py --model_path {onnx_model} --img_show

# Inference with RKNN model
python yolov8_seg.py --model_path {rknn_model} --target {target_platform} --img_show

# COCO mAP Test
python yolov8_seg.py --model_path {rknn_model} --target {target_platform} --anno_json {val_annotation} --img_folder {val_dataset}  --coco_map_test

Description:

  • {onnx_model / rknn_model} should be the model path.
  • {target_platform} Specify NPU platform name. Support Platform refer [here](#2 Current Support Platform).
  • {val_annotation} is the path of COCO val annotation.
  • {val_dataset} is the path of COCO val images.

6. Android Demo

Note: RK1808, RV1109, RV1126 does not support Android.

6.1 Compile and Build

Please refer to the Compilation_Environment_Setup_Guide document to setup a cross-compilation environment and complete the compilation of C/C++ Demo.
Note: Please replace the model name with yolov8_seg.

6.2 Push demo files to device

With device connected via USB port, push demo files to devices:

adb root
adb remount
adb push install/<TARGET_PLATFORM>_android_<ARCH>/rknn_yolov8_seg_demo/ /data/

6.3 Run demo

adb shell
cd /data/rknn_yolov8_seg_demo

export LD_LIBRARY_PATH=./lib
./rknn_yolov8_seg_demo model/yolov8-seg.rknn model/bus.jpg

  • After running, the result was saved as out.png. To check the result on host PC, pull back result referring to the following command:

    adb pull /data/rknn_yolov8_seg_demo/out.png

7. Linux Demo

7.1 Compile and Build

Please refer to the Compilation_Environment_Setup_Guide document to setup a cross-compilation environment and complete the compilation of C/C++ Demo.
Note: Please replace the model name with yolov8_seg.

7.2 Push demo files to device

  • If device connected via USB port, push demo files to devices:
adb push install/<TARGET_PLATFORM>_linux_<ARCH>/rknn_yolov8_seg_demo/ /userdata/
  • For other boards, use scp or other approaches to push all files under install/<TARGET_PLATFORM>_linux_<ARCH>/rknn_yolov8_seg_demo/ to userdata.

7.3 Run demo

adb shell
cd /userdata/rknn_yolov8_seg_demo

export LD_LIBRARY_PATH=./lib
./rknn_yolov8_seg_demo model/yolov8-seg.rknn model/bus.jpg

  • After running, the result was saved as out.png. To check the result on host PC, pull back result referring to the following command:

    adb pull /userdata/rknn_yolov8_seg_demo/out.png

8. Expected Results

This example will print the labels and corresponding scores of the test image detect results, as follows:

bus @ (87 137 553 439) 0.915
person @ (109 236 225 534) 0.904
person @ (211 241 283 508) 0.873
person @ (477 234 559 519) 0.869
person @ (79 327 125 514) 0.540
tie @ (248 284 259 310) 0.274

  • Note: Different platforms, different versions of tools and drivers may have slightly different results.