diff --git a/examples/vision/detection/scaledyolov4/README.md b/examples/vision/detection/scaledyolov4/README.md old mode 100644 new mode 100755 index 2256db66eb..b177b23197 --- a/examples/vision/detection/scaledyolov4/README.md +++ b/examples/vision/detection/scaledyolov4/README.md @@ -11,7 +11,7 @@ English | [简体中文](README_CN.md) Visit the official [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4) github repository, follow the guidelines to download the `scaledyolov4.pt` model, and employ `models/export.py` to get the file in `onnx` format. If you have any problems with the exported `onnx` model, refer to [ScaledYOLOv4#401](https://github.com/WongKinYiu/ScaledYOLOv4/issues/401) for solution. - + ```bash # Download the ScaledYOLOv4 model file @@ -38,8 +38,6 @@ For developers' testing, models exported by ScaledYOLOv4 are provided below. Dev | [ScaledYOLOv4-P6+BoF](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p6_.onnx) | 487MB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4),GPL-3.0 License | | [ScaledYOLOv4-P7](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p7.onnx) | 1.1GB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4),GPL-3.0 License | - - ## Detailed Deployment Documents - [Python Deployment](python) @@ -48,4 +46,4 @@ For developers' testing, models exported by ScaledYOLOv4 are provided below. Dev ## Release Note -- Document and code are based on [ScaledYOLOv4 CommitID: 6768003](https://github.com/WongKinYiu/ScaledYOLOv4/commit/676800364a3446900b9e8407bc880ea2127b3415) +- Document and code are based on [ScaledYOLOv4 CommitID: 6768003](https://github.com/WongKinYiu/ScaledYOLOv4/commit/676800364a3446900b9e8407bc880ea2127b3415) diff --git a/examples/vision/detection/yolor/README.md b/examples/vision/detection/yolor/README.md old mode 100644 new mode 100755 index 09972feda5..1665f304ab --- a/examples/vision/detection/yolor/README.md +++ b/examples/vision/detection/yolor/README.md @@ -36,7 +36,6 @@ For developers' testing, models exported by YOLOR are provided below. Developers | [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-570-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor),GPL-3.0 License | | [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-573-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor),GPL-3.0 License | - ## Detailed Deployment Documents - [Python Deployment](python) diff --git a/examples/vision/detection/yolov5/README.md b/examples/vision/detection/yolov5/README.md old mode 100644 new mode 100755 index 918b7d2537..9025a64640 --- a/examples/vision/detection/yolov5/README.md +++ b/examples/vision/detection/yolov5/README.md @@ -6,7 +6,6 @@ English | [简体中文](README_CN.md) - (1)The *.onnx provided by [Official Repository](https://github.com/ultralytics/yolov5/releases/tag/v7.0) can be deployed directly; - (2)The YOLOv5 v7.0 model trained by personal data should employ `export.py` in [YOLOv5](https://github.com/ultralytics/yolov5) to export the ONNX files for deployment. - ## Download Pre-trained ONNX Model For developers' testing, models exported by YOLOv5 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository) @@ -27,4 +26,4 @@ For developers' testing, models exported by YOLOv5 are provided below. Developer ## Release Note -- Document and code are based on [YOLOv5 v7.0](https://github.com/ultralytics/yolov5/tree/v7.0) +- Document and code are based on [YOLOv5 v7.0](https://github.com/ultralytics/yolov5/tree/v7.0) diff --git a/examples/vision/detection/yolov5lite/README.md b/examples/vision/detection/yolov5lite/README.md old mode 100644 new mode 100755 index 8487cb93b8..d245071586 --- a/examples/vision/detection/yolov5lite/README.md +++ b/examples/vision/detection/yolov5lite/README.md @@ -60,7 +60,6 @@ For developers' testing, models exported by YOLOv5Lite are provided below. Devel | [YOLOv5Lite-c](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-c-sim-512.onnx) | 18MB | 50.9% | This model file is sourced from[YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite),GPL-3.0 License | | [YOLOv5Lite-g](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-g-sim-640.onnx) | 21MB | 57.6% | This model file is sourced from [YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite),GPL-3.0 License | - ## Detailed Deployment Documents - [Python Deployment](python) @@ -69,4 +68,4 @@ For developers' testing, models exported by YOLOv5Lite are provided below. Devel ## Release Note -- Document and code are based on [YOLOv5-Lite v1.4](https://github.com/ppogg/YOLOv5-Lite/releases/tag/v1.4) +- Document and code are based on [YOLOv5-Lite v1.4](https://github.com/ppogg/YOLOv5-Lite/releases/tag/v1.4) diff --git a/examples/vision/detection/yolov6/README.md b/examples/vision/detection/yolov6/README.md old mode 100644 new mode 100755 index ae707ced85..6e4ede6eeb --- a/examples/vision/detection/yolov6/README.md +++ b/examples/vision/detection/yolov6/README.md @@ -8,8 +8,6 @@ English | [简体中文](README_CN.md) - (1)The *.onnx provided by [Official Repository](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) can directly conduct deployemnt; - (2)Personal models trained by developers should export the ONNX model. Refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. - - ## Download Pre-trained ONNX Model For developers' testing, models exported by YOLOv6 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository) @@ -20,8 +18,6 @@ For developers' testing, models exported by YOLOv6 are provided below. Developer | [YOLOv6t](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6t.onnx) | 58MB | 41.3% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6),GPL-3.0 License | | [YOLOv6n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6n.onnx) | 17MB | 35.0% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6),GPL-3.0 License | - - ## Detailed Deployment Documents - [Python Deployment](python) @@ -30,4 +26,4 @@ For developers' testing, models exported by YOLOv6 are provided below. Developer ## Release Note -- Document and code are based on [YOLOv6 0.1.0 version](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) +- Document and code are based on [YOLOv6 0.1.0 version](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) diff --git a/examples/vision/detection/yolov7/README.md b/examples/vision/detection/yolov7/README.md old mode 100644 new mode 100755 index 153ab13d50..5a1b6b8dca --- a/examples/vision/detection/yolov7/README.md +++ b/examples/vision/detection/yolov7/README.md @@ -20,8 +20,6 @@ python models/export.py --grid --dynamic --weights PATH/TO/yolov7.pt python models/export.py --grid --dynamic --end2end --weights PATH/TO/yolov7.pt ``` -## Download the pre-trained ONNX model - To facilitate testing for developers, we provide below the models exported by YOLOv7, which developers can download and use directly. (The accuracy of the models in the table is sourced from the official library) | Model | Size | Accuracy | Note | diff --git a/examples/vision/detection/yolov7end2end_ort/README.md b/examples/vision/detection/yolov7end2end_ort/README.md old mode 100644 new mode 100755 index 32395203c8..4176223a76 --- a/examples/vision/detection/yolov7end2end_ort/README.md +++ b/examples/vision/detection/yolov7end2end_ort/README.md @@ -31,13 +31,11 @@ For developers' testing, models exported by YOLOv7End2EndORT are provided below. | [yolov7-d6-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-d6-end2end-ort-nms.onnx) | 511MB | 56.6% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7),GPL-3.0 License | | [yolov7-e6e-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-e6e-end2end-ort-nms.onnx) | 579MB | 56.8% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7),GPL-3.0 License | - ## Detailed Deployment Documents - [Python Deployment](python) - [C++ Deployment](cpp) - ## Release Note -- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) +- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) diff --git a/examples/vision/detection/yolov7end2end_trt/README.md b/examples/vision/detection/yolov7end2end_trt/README.md old mode 100644 new mode 100755 index 24a9dcf516..44e3c0b3a2 --- a/examples/vision/detection/yolov7end2end_trt/README.md +++ b/examples/vision/detection/yolov7end2end_trt/README.md @@ -6,8 +6,6 @@ The YOLOv7End2EndTRT deployment is based on [YOLOv7](https://github.com/WongKinY - (1)*.pt provided by [Official Repository](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1) should [Export the ONNX Model](#Export-the-ONNX-Model) to complete the deployment. The deployment of *.trt and *.pose models is not supported. - (2)The YOLOv7 model trained by personal data should [Export the ONNX Model](#%E5%AF%BC%E5%87%BAONNX%E6%A8%A1%E5%9E%8B). Please refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. - - ## Export the ONNX Model ```bash @@ -37,7 +35,6 @@ For developers' testing, models exported by YOLOv7End2EndTRT are provided below. - [Python Deployment](python) - [C++ Deployement](cpp) - ## Release Note -- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) +- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) diff --git a/examples/vision/detection/yolov8/README.md b/examples/vision/detection/yolov8/README.md new file mode 100755 index 0000000000..245b46d309 --- /dev/null +++ b/examples/vision/detection/yolov8/README.md @@ -0,0 +1,29 @@ +English | [简体中文](README_CN.md) + +# YOLOv8 Ready-to-deploy Model + +- The deployment of the YOLOv8 model is based on [YOLOv8](https://github.com/ultralytics/ultralytics) and [Pre-trained Model Based on COCO](https://github.com/ultralytics/ultralytics) + - (1)The *.onnx provided by [Official Repository](https://github.com/ultralytics/ultralytics) can be deployed directly; + - (2)The YOLOv8 model trained by personal data should employ `export.py` in [YOLOv8](https://github.com/ultralytics/ultralytics) to export the ONNX files for deployment. + +## Download Pre-trained ONNX Model + +For developers' testing, models exported by YOLOv8 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository) +| Model | Size | Accuracy | Note | +|:---------------------------------------------------------------- |:----- |:----- |:---- | +| [YOLOv8n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8n.onnx) | 12.1MB | 37.3% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8s](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx) | 42.6MB | 44.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8m](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8m.onnx) | 98.8MB | 50.2% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8l](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8l.onnx) | 166.7MB | 52.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8x](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8x.onnx) | 260.3MB | 53.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | + + +## Detailed Deployment Documents + +- [Python Deployment](python) +- [C++ Deployment](cpp) +- [Serving Deployment](serving) + +## Release Note + +- Document and code are based on [YOLOv8](https://github.com/ultralytics/ultralytics) diff --git a/examples/vision/detection/yolov8/README_CN.md b/examples/vision/detection/yolov8/README_CN.md new file mode 100644 index 0000000000..a6635a59b4 --- /dev/null +++ b/examples/vision/detection/yolov8/README_CN.md @@ -0,0 +1,29 @@ +[English](README.md) | 简体中文 +# YOLOv8准备部署模型 + +- YOLOv8部署模型实现来自[YOLOv8](https://github.com/ultralytics/ultralytics),和[基于COCO的预训练模型](https://github.com/ultralytics/ultralytics) + - (1)[官方库](https://github.com/ultralytics/ultralytics)提供的*.onnx可直接进行部署; + - (2)开发者基于自己数据训练的YOLOv8模型,可使用[YOLOv8](https://github.com/ultralytics/ultralytics)中的`export.py`导出ONNX文件后,完成部署。 + + +## 下载预训练ONNX模型 + +为了方便开发者的测试,下面提供了YOLOv8导出的各系列模型,开发者可直接下载使用。(下表中模型的精度来源于源官方库) +| 模型 | 大小 | 精度 | 备注 | +|:---------------------------------------------------------------- |:----- |:----- |:---- | +| [YOLOv8n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8n.onnx) | 12.1MB | 37.3% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8s](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx) | 42.6MB | 44.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8m](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8m.onnx) | 98.8MB | 50.2% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8l](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8l.onnx) | 166.7MB | 52.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | +| [YOLOv8x](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8x.onnx) | 260.3MB | 53.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics),GPL-3.0 License | + + +## 详细部署文档 + +- [Python部署](python) +- [C++部署](cpp) +- [服务化部署](serving) + +## 版本说明 + +- 本版本文档和代码基于[YOLOv8](https://github.com/ultralytics/ultralytics) 编写 diff --git a/examples/vision/detection/yolov8/cpp/CMakeLists.txt b/examples/vision/detection/yolov8/cpp/CMakeLists.txt new file mode 100755 index 0000000000..6610d04d2a --- /dev/null +++ b/examples/vision/detection/yolov8/cpp/CMakeLists.txt @@ -0,0 +1,14 @@ +PROJECT(infer_demo C CXX) +CMAKE_MINIMUM_REQUIRED (VERSION 3.10) + +# Specify the fastdeploy library path after downloading and decompression +option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.") + +include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake) + +# Add FastDeploy dependent header files +include_directories(${FASTDEPLOY_INCS}) + +add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc) +# Add FastDeploy library dependencies +target_link_libraries(infer_demo ${FASTDEPLOY_LIBS}) diff --git a/examples/vision/detection/yolov8/cpp/README_CN.md b/examples/vision/detection/yolov8/cpp/README_CN.md new file mode 100644 index 0000000000..d1d87ccc5f --- /dev/null +++ b/examples/vision/detection/yolov8/cpp/README_CN.md @@ -0,0 +1,90 @@ +[English](README.md) | 简体中文 +# YOLOv8 C++部署示例 + +本目录下提供`infer.cc`快速完成YOLOv8在CPU/GPU,以及GPU上通过TensorRT加速部署的示例。 + +在部署前,需确认以下两个步骤 + +- 1. 软硬件环境满足要求,参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) +- 2. 根据开发环境,下载预编译部署库和samples代码,参考[FastDeploy预编译库](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) + +以Linux上CPU推理为例,在本目录执行如下命令即可完成编译测试,支持此模型需保证FastDeploy版本1.0.3以上(x.x.x>=1.0.3) + +```bash +mkdir build +cd build +# 下载 FastDeploy 预编译库,用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用 +wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz +tar xvf fastdeploy-linux-x64-x.x.x.tgz +cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x +make -j + +# 1. 下载官方转换好的 YOLOv8 ONNX 模型文件和测试图片 +wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx +wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg + +# CPU推理 +./infer_demo yolov8s.onnx 000000014439.jpg 0 +# GPU推理 +./infer_demo yolov8s.onnx 000000014439.jpg 1 +# GPU上TensorRT推理 +./infer_demo yolov8s.onnx 000000014439.jpg 2 +``` +运行完成可视化结果如下图所示 + + + +以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考: +- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md) + +如果用户使用华为昇腾NPU部署, 请参考以下方式在部署前初始化部署环境: +- [如何使用华为昇腾NPU部署](../../../../../docs/cn/faq/use_sdk_on_ascend.md) + +## YOLOv8 C++接口 + +### YOLOv8类 + +```c++ +fastdeploy::vision::detection::YOLOv8( + const string& model_file, + const string& params_file = "", + const RuntimeOption& runtime_option = RuntimeOption(), + const ModelFormat& model_format = ModelFormat::ONNX) +``` + +YOLOv8模型加载和初始化,其中model_file为导出的ONNX模型格式。 + +**参数** + +> * **model_file**(str): 模型文件路径 +> * **params_file**(str): 参数文件路径,当模型格式为ONNX时,此参数传入空字符串即可 +> * **runtime_option**(RuntimeOption): 后端推理配置,默认为None,即采用默认配置 +> * **model_format**(ModelFormat): 模型格式,默认为ONNX格式 + +#### Predict函数 + +> ```c++ +> YOLOv8::Predict(cv::Mat* im, DetectionResult* result) +> ``` +> +> 模型预测接口,输入图像直接输出检测结果。 +> +> **参数** +> +> > * **im**: 输入图像,注意需为HWC,BGR格式 +> > * **result**: 检测结果,包括检测框,各个框的置信度, DetectionResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/) + +### 类成员变量 +#### 预处理参数 +用户可按照自己的实际需求,修改下列预处理参数,从而影响最终的推理和部署效果 + +> > * **size**(vector<int>): 通过此参数修改预处理过程中resize的大小,包含两个整型元素,表示[width, height], 默认值为[640, 640] +> > * **padding_value**(vector<float>): 通过此参数可以修改图片在resize时候做填充(padding)的值, 包含三个浮点型元素, 分别表示三个通道的值, 默认值为[114, 114, 114] +> > * **is_no_pad**(bool): 通过此参数让图片是否通过填充的方式进行resize, `is_no_pad=ture` 表示不使用填充的方式,默认值为`is_no_pad=false` +> > * **is_mini_pad**(bool): 通过此参数可以将resize之后图像的宽高这是为最接近`size`成员变量的值, 并且满足填充的像素大小是可以被`stride`成员变量整除的。默认值为`is_mini_pad=false` +> > * **stride**(int): 配合`stris_mini_pad`成员变量使用, 默认值为`stride=32` + +- [模型介绍](../../) +- [Python部署](../python) +- [视觉模型预测结果](../../../../../docs/api/vision_results/) +- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md) diff --git a/examples/vision/detection/yolov8/cpp/infer.cc b/examples/vision/detection/yolov8/cpp/infer.cc new file mode 100755 index 0000000000..e517634d8e --- /dev/null +++ b/examples/vision/detection/yolov8/cpp/infer.cc @@ -0,0 +1,105 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/vision.h" + +void CpuInfer(const std::string& model_file, const std::string& image_file) { + auto model = fastdeploy::vision::detection::YOLOv8(model_file); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + std::cout << res.Str() << std::endl; + + auto vis_im = fastdeploy::vision::VisDetection(im, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +void GpuInfer(const std::string& model_file, const std::string& image_file) { + auto option = fastdeploy::RuntimeOption(); + option.UseGpu(); + auto model = fastdeploy::vision::detection::YOLOv8(model_file, "", option); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + std::cout << res.Str() << std::endl; + + auto vis_im = fastdeploy::vision::VisDetection(im, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +void TrtInfer(const std::string& model_file, const std::string& image_file) { + auto option = fastdeploy::RuntimeOption(); + option.UseGpu(); + option.UseTrtBackend(); + option.SetTrtInputShape("images", {1, 3, 640, 640}); + auto model = fastdeploy::vision::detection::YOLOv8(model_file, "", option); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + std::cout << res.Str() << std::endl; + + auto vis_im = fastdeploy::vision::VisDetection(im, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +int main(int argc, char* argv[]) { + if (argc < 4) { + std::cout << "Usage: infer_demo path/to/model path/to/image run_option, " + "e.g ./infer_model ./yolov8s.onnx ./test.jpeg 0" + << std::endl; + std::cout << "The data type of run_option is int, 0: run with cpu; 1: run " + "with gpu; 2: run with gpu and use tensorrt backend." + << std::endl; + return -1; + } + + if (std::atoi(argv[3]) == 0) { + CpuInfer(argv[1], argv[2]); + } else if (std::atoi(argv[3]) == 1) { + GpuInfer(argv[1], argv[2]); + } else if (std::atoi(argv[3]) == 2) { + TrtInfer(argv[1], argv[2]); + } + return 0; +} \ No newline at end of file diff --git a/examples/vision/detection/yolov8/python/README_CN.md b/examples/vision/detection/yolov8/python/README_CN.md new file mode 100644 index 0000000000..ae19ec7271 --- /dev/null +++ b/examples/vision/detection/yolov8/python/README_CN.md @@ -0,0 +1,78 @@ +[English](README.md) | 简体中文 +# YOLOv8 Python部署示例 + +在部署前,需确认以下两个步骤 + +- 1. 软硬件环境满足要求,参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) +- 2. FastDeploy Python whl包安装,参考[FastDeploy Python安装](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) + +本目录下提供`infer.py`快速完成YOLOv8在CPU/GPU,以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成 + +```bash +#下载部署示例代码 +git clone https://github.com/PaddlePaddle/FastDeploy.git +cd examples/vision/detection/yolov8/python/ + +#下载yolov8模型文件和测试图片 +wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov8.onnx +wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg + +# CPU推理 +python infer.py --model yolov8.onnx --image 000000014439.jpg --device cpu +# GPU推理 +python infer.py --model yolov8.onnx --image 000000014439.jpg --device gpu +# GPU上使用TensorRT推理 +python infer.py --model yolov8.onnx --image 000000014439.jpg --device gpu --use_trt True +``` + +运行完成可视化结果如下图所示 + + + +## YOLOv8 Python接口 + +```python +fastdeploy.vision.detection.YOLOv8(model_file, params_file=None, runtime_option=None, model_format=ModelFormat.ONNX) +``` + +YOLOv8模型加载和初始化,其中model_file为导出的ONNX模型格式 + +**参数** + +> * **model_file**(str): 模型文件路径 +> * **params_file**(str): 参数文件路径,当模型格式为ONNX格式时,此参数无需设定 +> * **runtime_option**(RuntimeOption): 后端推理配置,默认为None,即采用默认配置 +> * **model_format**(ModelFormat): 模型格式,默认为ONNX + +### predict函数 + +> ```python +> YOLOv8.predict(image_data) +> ``` +> +> 模型预测结口,输入图像直接输出检测结果。 +> +> **参数** +> +> > * **image_data**(np.ndarray): 输入数据,注意需为HWC,BGR格式 + +> **返回** +> +> > 返回`fastdeploy.vision.DetectionResult`结构体,结构体说明参考文档[视觉模型预测结果](../../../../../docs/api/vision_results/) + +### 类成员属性 +#### 预处理参数 +用户可按照自己的实际需求,修改下列预处理参数,从而影响最终的推理和部署效果 + +> > * **size**(list[int]): 通过此参数修改预处理过程中resize的大小,包含两个整型元素,表示[width, height], 默认值为[640, 640] +> > * **padding_value**(list[float]): 通过此参数可以修改图片在resize时候做填充(padding)的值, 包含三个浮点型元素, 分别表示三个通道的值, 默认值为[114, 114, 114] +> > * **is_no_pad**(bool): 通过此参数让图片是否通过填充的方式进行resize, `is_no_pad=True` 表示不使用填充的方式,默认值为`is_no_pad=False` +> > * **is_mini_pad**(bool): 通过此参数可以将resize之后图像的宽高这是为最接近`size`成员变量的值, 并且满足填充的像素大小是可以被`stride`成员变量整除的。默认值为`is_mini_pad=False` +> > * **stride**(int): 配合`stris_mini_padide`成员变量使用, 默认值为`stride=32` + +## 其它文档 + +- [YOLOv8 模型介绍](..) +- [YOLOv8 C++部署](../cpp) +- [模型预测结果说明](../../../../../docs/api/vision_results/) +- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md) diff --git a/examples/vision/detection/yolov8/python/infer.py b/examples/vision/detection/yolov8/python/infer.py new file mode 100755 index 0000000000..9673b3da0d --- /dev/null +++ b/examples/vision/detection/yolov8/python/infer.py @@ -0,0 +1,58 @@ +import fastdeploy as fd +import cv2 +import os + + +def parse_arguments(): + import argparse + import ast + parser = argparse.ArgumentParser() + parser.add_argument("--model", default=None, help="Path of yolov8 model.") + parser.add_argument( + "--image", default=None, help="Path of test image file.") + parser.add_argument( + "--device", + type=str, + default='cpu', + help="Type of inference device, support 'cpu' or 'gpu' or 'kunlunxin'.") + parser.add_argument( + "--use_trt", + type=ast.literal_eval, + default=False, + help="Wether to use tensorrt.") + return parser.parse_args() + + +def build_option(args): + option = fd.RuntimeOption() + + if args.device.lower() == "gpu": + option.use_gpu() + + if args.device.lower() == "ascend": + option.use_ascend() + + if args.use_trt: + option.use_trt_backend() + option.set_trt_input_shape("images", [1, 3, 640, 640]) + return option + + +args = parse_arguments() + +# Configure runtime, load model +runtime_option = build_option(args) +model = fd.vision.detection.YOLOv8(args.model, runtime_option=runtime_option) + +# Predicting image +if args.image is None: + image = fd.utils.get_detection_test_image() +else: + image = args.image +im = cv2.imread(image) +result = model.predict(im) + +# Visualization +vis_im = fd.vision.vis_detection(im, result) +cv2.imwrite("visualized_result.jpg", vis_im) +print("Visualized result save in ./visualized_result.jpg") diff --git a/fastdeploy/vision.h b/fastdeploy/vision.h index 867de58cb1..8788e889dc 100755 --- a/fastdeploy/vision.h +++ b/fastdeploy/vision.h @@ -29,6 +29,7 @@ #include "fastdeploy/vision/detection/contrib/yolov7/yolov7.h" #include "fastdeploy/vision/detection/contrib/yolov7end2end_ort.h" #include "fastdeploy/vision/detection/contrib/yolov7end2end_trt.h" +#include "fastdeploy/vision/detection/contrib/yolov8/yolov8.h" #include "fastdeploy/vision/detection/contrib/yolox.h" #include "fastdeploy/vision/detection/contrib/rknpu2/model.h" #include "fastdeploy/vision/detection/ppdet/model.h" diff --git a/fastdeploy/vision/detection/contrib/yolov5/yolov5_pybind.cc b/fastdeploy/vision/detection/contrib/yolov5/yolov5_pybind.cc old mode 100755 new mode 100644 index c88270c041..c61d5613bd --- a/fastdeploy/vision/detection/contrib/yolov5/yolov5_pybind.cc +++ b/fastdeploy/vision/detection/contrib/yolov5/yolov5_pybind.cc @@ -16,54 +16,82 @@ namespace fastdeploy { void BindYOLOv5(pybind11::module& m) { - pybind11::class_( - m, "YOLOv5Preprocessor") + pybind11::class_(m, + "YOLOv5Preprocessor") .def(pybind11::init<>()) - .def("run", [](vision::detection::YOLOv5Preprocessor& self, std::vector& im_list) { - std::vector images; - for (size_t i = 0; i < im_list.size(); ++i) { - images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i]))); - } - std::vector outputs; - std::vector>> ims_info; - if (!self.Run(&images, &outputs, &ims_info)) { - throw std::runtime_error("Failed to preprocess the input data in PaddleClasPreprocessor."); - } - for (size_t i = 0; i < outputs.size(); ++i) { - outputs[i].StopSharing(); - } - return make_pair(outputs, ims_info); - }) - .def_property("size", &vision::detection::YOLOv5Preprocessor::GetSize, &vision::detection::YOLOv5Preprocessor::SetSize) - .def_property("padding_value", &vision::detection::YOLOv5Preprocessor::GetPaddingValue, &vision::detection::YOLOv5Preprocessor::SetPaddingValue) - .def_property("is_scale_up", &vision::detection::YOLOv5Preprocessor::GetScaleUp, &vision::detection::YOLOv5Preprocessor::SetScaleUp) - .def_property("is_mini_pad", &vision::detection::YOLOv5Preprocessor::GetMiniPad, &vision::detection::YOLOv5Preprocessor::SetMiniPad) - .def_property("stride", &vision::detection::YOLOv5Preprocessor::GetStride, &vision::detection::YOLOv5Preprocessor::SetStride); + .def( + "run", + [](vision::detection::YOLOv5Preprocessor& self, + std::vector& im_list) { + std::vector images; + for (size_t i = 0; i < im_list.size(); ++i) { + images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i]))); + } + std::vector outputs; + std::vector>> ims_info; + if (!self.Run(&images, &outputs, &ims_info)) { + throw std::runtime_error( + "Failed to preprocess the input data in YOLOv5Preprocessor."); + } + for (size_t i = 0; i < outputs.size(); ++i) { + outputs[i].StopSharing(); + } + return make_pair(outputs, ims_info); + }) + .def_property("size", &vision::detection::YOLOv5Preprocessor::GetSize, + &vision::detection::YOLOv5Preprocessor::SetSize) + .def_property("padding_value", + &vision::detection::YOLOv5Preprocessor::GetPaddingValue, + &vision::detection::YOLOv5Preprocessor::SetPaddingValue) + .def_property("is_scale_up", + &vision::detection::YOLOv5Preprocessor::GetScaleUp, + &vision::detection::YOLOv5Preprocessor::SetScaleUp) + .def_property("is_mini_pad", + &vision::detection::YOLOv5Preprocessor::GetMiniPad, + &vision::detection::YOLOv5Preprocessor::SetMiniPad) + .def_property("stride", &vision::detection::YOLOv5Preprocessor::GetStride, + &vision::detection::YOLOv5Preprocessor::SetStride); pybind11::class_( m, "YOLOv5Postprocessor") .def(pybind11::init<>()) - .def("run", [](vision::detection::YOLOv5Postprocessor& self, std::vector& inputs, - const std::vector>>& ims_info) { - std::vector results; - if (!self.Run(inputs, &results, ims_info)) { - throw std::runtime_error("Failed to postprocess the runtime result in YOLOv5Postprocessor."); - } - return results; - }) - .def("run", [](vision::detection::YOLOv5Postprocessor& self, std::vector& input_array, - const std::vector>>& ims_info) { - std::vector results; - std::vector inputs; - PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true); - if (!self.Run(inputs, &results, ims_info)) { - throw std::runtime_error("Failed to postprocess the runtime result in YOLOv5Postprocessor."); - } - return results; - }) - .def_property("conf_threshold", &vision::detection::YOLOv5Postprocessor::GetConfThreshold, &vision::detection::YOLOv5Postprocessor::SetConfThreshold) - .def_property("nms_threshold", &vision::detection::YOLOv5Postprocessor::GetNMSThreshold, &vision::detection::YOLOv5Postprocessor::SetNMSThreshold) - .def_property("multi_label", &vision::detection::YOLOv5Postprocessor::GetMultiLabel, &vision::detection::YOLOv5Postprocessor::SetMultiLabel); + .def("run", + [](vision::detection::YOLOv5Postprocessor& self, + std::vector& inputs, + const std::vector>>& + ims_info) { + std::vector results; + if (!self.Run(inputs, &results, ims_info)) { + throw std::runtime_error( + "Failed to postprocess the runtime result in " + "YOLOv5Postprocessor."); + } + return results; + }) + .def("run", + [](vision::detection::YOLOv5Postprocessor& self, + std::vector& input_array, + const std::vector>>& + ims_info) { + std::vector results; + std::vector inputs; + PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true); + if (!self.Run(inputs, &results, ims_info)) { + throw std::runtime_error( + "Failed to postprocess the runtime result in " + "YOLOv5Postprocessor."); + } + return results; + }) + .def_property("conf_threshold", + &vision::detection::YOLOv5Postprocessor::GetConfThreshold, + &vision::detection::YOLOv5Postprocessor::SetConfThreshold) + .def_property("nms_threshold", + &vision::detection::YOLOv5Postprocessor::GetNMSThreshold, + &vision::detection::YOLOv5Postprocessor::SetNMSThreshold) + .def_property("multi_label", + &vision::detection::YOLOv5Postprocessor::GetMultiLabel, + &vision::detection::YOLOv5Postprocessor::SetMultiLabel); pybind11::class_(m, "YOLOv5") .def(pybind11::init& data) { - std::vector images; - for (size_t i = 0; i < data.size(); ++i) { - images.push_back(PyArrayToCvMat(data[i])); - } - std::vector results; - self.BatchPredict(images, &results); - return results; - }) - .def_property_readonly("preprocessor", &vision::detection::YOLOv5::GetPreprocessor) - .def_property_readonly("postprocessor", &vision::detection::YOLOv5::GetPostprocessor); + .def("batch_predict", + [](vision::detection::YOLOv5& self, + std::vector& data) { + std::vector images; + for (size_t i = 0; i < data.size(); ++i) { + images.push_back(PyArrayToCvMat(data[i])); + } + std::vector results; + self.BatchPredict(images, &results); + return results; + }) + .def_property_readonly("preprocessor", + &vision::detection::YOLOv5::GetPreprocessor) + .def_property_readonly("postprocessor", + &vision::detection::YOLOv5::GetPostprocessor); } } // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/postprocessor.cc b/fastdeploy/vision/detection/contrib/yolov8/postprocessor.cc new file mode 100755 index 0000000000..77974cdd68 --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/postprocessor.cc @@ -0,0 +1,143 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/vision/detection/contrib/yolov8/postprocessor.h" +#include "fastdeploy/vision/utils/utils.h" + +namespace fastdeploy { +namespace vision { +namespace detection { + +YOLOv8Postprocessor::YOLOv8Postprocessor() { + conf_threshold_ = 0.25; + nms_threshold_ = 0.5; + multi_label_ = true; + max_wh_ = 7680.0; +} + +bool YOLOv8Postprocessor::Run( + const std::vector& tensors, std::vector* results, + const std::vector>>& ims_info) { + int batch = tensors[0].shape[0]; + // transpose + std::vector dim{0, 2, 1}; + FDTensor tensor_transpose; + function::Transpose(tensors[0], &tensor_transpose, dim); + + results->resize(batch); + + for (size_t bs = 0; bs < batch; ++bs) { + (*results)[bs].Clear(); + if (multi_label_) { + (*results)[bs].Reserve(tensor_transpose.shape[1] * + (tensor_transpose.shape[2] - 4)); + } else { + (*results)[bs].Reserve(tensor_transpose.shape[1]); + } + if (tensor_transpose.dtype != FDDataType::FP32) { + FDERROR << "Only support post process with float32 data." << std::endl; + return false; + } + const float* data = + reinterpret_cast(tensor_transpose.Data()) + + bs * tensor_transpose.shape[1] * tensor_transpose.shape[2]; + for (size_t i = 0; i < tensor_transpose.shape[1]; ++i) { + int s = i * tensor_transpose.shape[2]; + if (multi_label_) { + for (size_t j = 4; j < tensor_transpose.shape[2]; ++j) { + float confidence = data[s + j]; + // filter boxes by conf_threshold + if (confidence <= conf_threshold_) { + continue; + } + int32_t label_id = j - 4; + + // convert from [x, y, w, h] to [x1, y1, x2, y2] + (*results)[bs].boxes.emplace_back(std::array{ + data[s] - data[s + 2] / 2.0f + label_id * max_wh_, + data[s + 1] - data[s + 3] / 2.0f + label_id * max_wh_, + data[s + 0] + data[s + 2] / 2.0f + label_id * max_wh_, + data[s + 1] + data[s + 3] / 2.0f + label_id * max_wh_}); + (*results)[bs].label_ids.push_back(label_id); + (*results)[bs].scores.push_back(confidence); + } + } else { + const float* max_class_score = std::max_element( + data + s + 4, data + s + tensor_transpose.shape[2]); + float confidence = *max_class_score; + // filter boxes by conf_threshold + if (confidence <= conf_threshold_) { + continue; + } + int32_t label_id = std::distance(data + s + 4, max_class_score); + // convert from [x, y, w, h] to [x1, y1, x2, y2] + (*results)[bs].boxes.emplace_back(std::array{ + data[s] - data[s + 2] / 2.0f + label_id * max_wh_, + data[s + 1] - data[s + 3] / 2.0f + label_id * max_wh_, + data[s + 0] + data[s + 2] / 2.0f + label_id * max_wh_, + data[s + 1] + data[s + 3] / 2.0f + label_id * max_wh_}); + (*results)[bs].label_ids.push_back(label_id); + (*results)[bs].scores.push_back(confidence); + } + } + + if ((*results)[bs].boxes.size() == 0) { + return true; + } + + utils::NMS(&((*results)[bs]), nms_threshold_); + + // scale the boxes to the origin image shape + auto iter_out = ims_info[bs].find("output_shape"); + auto iter_ipt = ims_info[bs].find("input_shape"); + FDASSERT(iter_out != ims_info[bs].end() && iter_ipt != ims_info[bs].end(), + "Cannot find input_shape or output_shape from im_info."); + float out_h = iter_out->second[0]; + float out_w = iter_out->second[1]; + float ipt_h = iter_ipt->second[0]; + float ipt_w = iter_ipt->second[1]; + float scale = std::min(out_h / ipt_h, out_w / ipt_w); + float pad_h = (out_h - ipt_h * scale) / 2; + float pad_w = (out_w - ipt_w * scale) / 2; + for (size_t i = 0; i < (*results)[bs].boxes.size(); ++i) { + int32_t label_id = ((*results)[bs].label_ids)[i]; + // clip box + (*results)[bs].boxes[i][0] = + (*results)[bs].boxes[i][0] - max_wh_ * label_id; + (*results)[bs].boxes[i][1] = + (*results)[bs].boxes[i][1] - max_wh_ * label_id; + (*results)[bs].boxes[i][2] = + (*results)[bs].boxes[i][2] - max_wh_ * label_id; + (*results)[bs].boxes[i][3] = + (*results)[bs].boxes[i][3] - max_wh_ * label_id; + (*results)[bs].boxes[i][0] = + std::max(((*results)[bs].boxes[i][0] - pad_w) / scale, 0.0f); + (*results)[bs].boxes[i][1] = + std::max(((*results)[bs].boxes[i][1] - pad_h) / scale, 0.0f); + (*results)[bs].boxes[i][2] = + std::max(((*results)[bs].boxes[i][2] - pad_w) / scale, 0.0f); + (*results)[bs].boxes[i][3] = + std::max(((*results)[bs].boxes[i][3] - pad_h) / scale, 0.0f); + (*results)[bs].boxes[i][0] = std::min((*results)[bs].boxes[i][0], ipt_w); + (*results)[bs].boxes[i][1] = std::min((*results)[bs].boxes[i][1], ipt_h); + (*results)[bs].boxes[i][2] = std::min((*results)[bs].boxes[i][2], ipt_w); + (*results)[bs].boxes[i][3] = std::min((*results)[bs].boxes[i][3], ipt_h); + } + } + return true; +} + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/postprocessor.h b/fastdeploy/vision/detection/contrib/yolov8/postprocessor.h new file mode 100755 index 0000000000..2c7d1b9bd5 --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/postprocessor.h @@ -0,0 +1,75 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once +#include "fastdeploy/vision/common/processors/transform.h" +#include "fastdeploy/vision/common/result.h" + +namespace fastdeploy { +namespace vision { + +namespace detection { +/*! @brief Postprocessor object for YOLOv8 serials model. + */ +class FASTDEPLOY_DECL YOLOv8Postprocessor { + public: + /** \brief Create a postprocessor instance for YOLOv8 serials model + */ + YOLOv8Postprocessor(); + + /** \brief Process the result of runtime and fill to DetectionResult structure + * + * \param[in] tensors The inference result from runtime + * \param[in] result The output result of detection + * \param[in] ims_info The shape info list, record input_shape and output_shape + * \return true if the postprocess successed, otherwise false + */ + bool Run(const std::vector& tensors, + std::vector* results, + const std::vector>>& ims_info); + + /// Set conf_threshold, default 0.25 + void SetConfThreshold(const float& conf_threshold) { + conf_threshold_ = conf_threshold; + } + + /// Get conf_threshold, default 0.25 + float GetConfThreshold() const { return conf_threshold_; } + + /// Set nms_threshold, default 0.5 + void SetNMSThreshold(const float& nms_threshold) { + nms_threshold_ = nms_threshold; + } + + /// Get nms_threshold, default 0.5 + float GetNMSThreshold() const { return nms_threshold_; } + + /// Set multi_label, set true for eval, default true + void SetMultiLabel(bool multi_label) { + multi_label_ = multi_label; + } + + /// Get multi_label, default true + bool GetMultiLabel() const { return multi_label_; } + + protected: + float conf_threshold_; + float nms_threshold_; + bool multi_label_; + float max_wh_; +}; + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/preprocessor.cc b/fastdeploy/vision/detection/contrib/yolov8/preprocessor.cc new file mode 100644 index 0000000000..1c6d9f62c8 --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/preprocessor.cc @@ -0,0 +1,119 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/vision/detection/contrib/yolov8/preprocessor.h" +#include "fastdeploy/function/concat.h" + +namespace fastdeploy { +namespace vision { +namespace detection { + +YOLOv8Preprocessor::YOLOv8Preprocessor() { + size_ = {640, 640}; + padding_value_ = {114.0, 114.0, 114.0}; + is_mini_pad_ = false; + is_no_pad_ = false; + is_scale_up_ = true; + stride_ = 32; + max_wh_ = 7680.0; +} + +void YOLOv8Preprocessor::LetterBox(FDMat* mat) { + float scale = + std::min(size_[1] * 1.0 / mat->Height(), size_[0] * 1.0 / mat->Width()); + if (!is_scale_up_) { + scale = std::min(scale, 1.0f); + } + + int resize_h = int(round(mat->Height() * scale)); + int resize_w = int(round(mat->Width() * scale)); + + int pad_w = size_[0] - resize_w; + int pad_h = size_[1] - resize_h; + if (is_mini_pad_) { + pad_h = pad_h % stride_; + pad_w = pad_w % stride_; + } else if (is_no_pad_) { + pad_h = 0; + pad_w = 0; + resize_h = size_[1]; + resize_w = size_[0]; + } + if (std::fabs(scale - 1.0f) > 1e-06) { + Resize::Run(mat, resize_w, resize_h); + } + if (pad_h > 0 || pad_w > 0) { + float half_h = pad_h * 1.0 / 2; + int top = int(round(half_h - 0.1)); + int bottom = int(round(half_h + 0.1)); + float half_w = pad_w * 1.0 / 2; + int left = int(round(half_w - 0.1)); + int right = int(round(half_w + 0.1)); + Pad::Run(mat, top, bottom, left, right, padding_value_); + } +} + +bool YOLOv8Preprocessor::Preprocess( + FDMat* mat, FDTensor* output, + std::map>* im_info) { + // Record the shape of image and the shape of preprocessed image + (*im_info)["input_shape"] = {static_cast(mat->Height()), + static_cast(mat->Width())}; + // yolov8's preprocess steps + // 1. letterbox + // 2. convert_and_permute(swap_rb=true) + LetterBox(mat); + std::vector alpha = {1.0f / 255.0f, 1.0f / 255.0f, 1.0f / 255.0f}; + std::vector beta = {0.0f, 0.0f, 0.0f}; + ConvertAndPermute::Run(mat, alpha, beta, true); + + // Record output shape of preprocessed image + (*im_info)["output_shape"] = {static_cast(mat->Height()), + static_cast(mat->Width())}; + + mat->ShareWithTensor(output); + output->ExpandDim(0); // reshape to n, h, w, c + return true; +} + +bool YOLOv8Preprocessor::Run( + std::vector* images, std::vector* outputs, + std::vector>>* ims_info) { + if (images->size() == 0) { + FDERROR << "The size of input images should be greater than 0." + << std::endl; + return false; + } + ims_info->resize(images->size()); + outputs->resize(1); + // Concat all the preprocessed data to a batch tensor + std::vector tensors(images->size()); + for (size_t i = 0; i < images->size(); ++i) { + if (!Preprocess(&(*images)[i], &tensors[i], &(*ims_info)[i])) { + FDERROR << "Failed to preprocess input image." << std::endl; + return false; + } + } + + if (tensors.size() == 1) { + (*outputs)[0] = std::move(tensors[0]); + } else { + function::Concat(tensors, &((*outputs)[0]), 0); + } + return true; +} + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/preprocessor.h b/fastdeploy/vision/detection/contrib/yolov8/preprocessor.h new file mode 100755 index 0000000000..9fe1c848dc --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/preprocessor.h @@ -0,0 +1,113 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once +#include "fastdeploy/vision/common/processors/transform.h" +#include "fastdeploy/vision/common/result.h" + +namespace fastdeploy { +namespace vision { + +namespace detection { +/*! @brief Preprocessor object for YOLOv8 serials model. + */ +class FASTDEPLOY_DECL YOLOv8Preprocessor { + public: + /** \brief Create a preprocessor instance for YOLOv8 serials model + */ + YOLOv8Preprocessor(); + + /** \brief Process the input image and prepare input tensors for runtime + * + * \param[in] images The input image data list, all the elements are returned by cv::imread() + * \param[in] outputs The output tensors which will feed in runtime + * \param[in] ims_info The shape info list, record input_shape and output_shape + * \return true if the preprocess successed, otherwise false + */ + bool Run(std::vector* images, std::vector* outputs, + std::vector>>* ims_info); + + /// Set target size, tuple of (width, height), default size = {640, 640} + void SetSize(const std::vector& size) { size_ = size; } + + /// Get target size, tuple of (width, height), default size = {640, 640} + std::vector GetSize() const { return size_; } + + /// Set padding value, size should be the same as channels + void SetPaddingValue(const std::vector& padding_value) { + padding_value_ = padding_value; + } + + /// Get padding value, size should be the same as channels + std::vector GetPaddingValue() const { return padding_value_; } + + /// Set is_scale_up, if is_scale_up is false, the input image only + /// can be zoom out, the maximum resize scale cannot exceed 1.0, default true + void SetScaleUp(bool is_scale_up) { + is_scale_up_ = is_scale_up; + } + + /// Get is_scale_up, default true + bool GetScaleUp() const { return is_scale_up_; } + + /// Set is_mini_pad, pad to the minimum rectange + /// which height and width is times of stride + void SetMiniPad(bool is_mini_pad) { + is_mini_pad_ = is_mini_pad; + } + + /// Get is_mini_pad, default false + bool GetMiniPad() const { return is_mini_pad_; } + + /// Set padding stride, only for mini_pad mode + void SetStride(int stride) { + stride_ = stride; + } + + /// Get padding stride, default 32 + bool GetStride() const { return stride_; } + + protected: + bool Preprocess(FDMat* mat, FDTensor* output, + std::map>* im_info); + + void LetterBox(FDMat* mat); + + // target size, tuple of (width, height), default size = {640, 640} + std::vector size_; + + // padding value, size should be the same as channels + std::vector padding_value_; + + // only pad to the minimum rectange which height and width is times of stride + bool is_mini_pad_; + + // while is_mini_pad = false and is_no_pad = true, + // will resize the image to the set size + bool is_no_pad_; + + // if is_scale_up is false, the input image only can be zoom out, + // the maximum resize scale cannot exceed 1.0 + bool is_scale_up_; + + // padding stride, for is_mini_pad + int stride_; + + // for offseting the boxes by classes when using NMS + float max_wh_; +}; + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/yolov8.cc b/fastdeploy/vision/detection/contrib/yolov8/yolov8.cc new file mode 100644 index 0000000000..53fdd7f75a --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/yolov8.cc @@ -0,0 +1,82 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/vision/detection/contrib/yolov8/yolov8.h" + +namespace fastdeploy { +namespace vision { +namespace detection { + +YOLOv8::YOLOv8(const std::string& model_file, const std::string& params_file, + const RuntimeOption& custom_option, + const ModelFormat& model_format) { + if (model_format == ModelFormat::ONNX) { + valid_cpu_backends = {Backend::OPENVINO, Backend::ORT}; + valid_gpu_backends = {Backend::ORT, Backend::TRT}; + } else { + valid_cpu_backends = {Backend::PDINFER, Backend::ORT, Backend::LITE}; + valid_gpu_backends = {Backend::PDINFER, Backend::ORT, Backend::TRT}; + } + runtime_option = custom_option; + runtime_option.model_format = model_format; + runtime_option.model_file = model_file; + runtime_option.params_file = params_file; + initialized = Initialize(); +} + +bool YOLOv8::Initialize() { + if (!InitRuntime()) { + FDERROR << "Failed to initialize fastdeploy backend." << std::endl; + return false; + } + return true; +} + +bool YOLOv8::Predict(const cv::Mat& im, DetectionResult* result) { + std::vector results; + if (!BatchPredict({im}, &results)) { + return false; + } + *result = std::move(results[0]); + return true; +} + +bool YOLOv8::BatchPredict(const std::vector& images, + std::vector* results) { + std::vector>> ims_info; + std::vector fd_images = WrapMat(images); + + if (!preprocessor_.Run(&fd_images, &reused_input_tensors_, &ims_info)) { + FDERROR << "Failed to preprocess the input image." << std::endl; + return false; + } + + reused_input_tensors_[0].name = InputInfoOfRuntime(0).name; + if (!Infer(reused_input_tensors_, &reused_output_tensors_)) { + FDERROR << "Failed to inference by runtime." << std::endl; + return false; + } + + if (!postprocessor_.Run(reused_output_tensors_, results, ims_info)) { + FDERROR << "Failed to postprocess the inference results by runtime." + << std::endl; + return false; + } + + return true; +} + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/yolov8.h b/fastdeploy/vision/detection/contrib/yolov8/yolov8.h new file mode 100755 index 0000000000..88ebf05dde --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/yolov8.h @@ -0,0 +1,76 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. //NOLINT +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include "fastdeploy/fastdeploy_model.h" +#include "fastdeploy/vision/detection/contrib/yolov8/preprocessor.h" +#include "fastdeploy/vision/detection/contrib/yolov8/postprocessor.h" + +namespace fastdeploy { +namespace vision { +namespace detection { +/*! @brief YOLOv8 model object used when to load a YOLOv8 model exported by YOLOv8. + */ +class FASTDEPLOY_DECL YOLOv8 : public FastDeployModel { + public: + /** \brief Set path of model file and the configuration of runtime. + * + * \param[in] model_file Path of model file, e.g ./yolov8.onnx + * \param[in] params_file Path of parameter file, e.g ppyoloe/model.pdiparams, if the model format is ONNX, this parameter will be ignored + * \param[in] custom_option RuntimeOption for inference, the default will use cpu, and choose the backend defined in "valid_cpu_backends" + * \param[in] model_format Model format of the loaded model, default is ONNX format + */ + YOLOv8(const std::string& model_file, const std::string& params_file = "", + const RuntimeOption& custom_option = RuntimeOption(), + const ModelFormat& model_format = ModelFormat::ONNX); + + std::string ModelName() const { return "yolov8"; } + + /** \brief Predict the detection result for an input image + * + * \param[in] img The input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format + * \param[in] result The output detection result will be writen to this structure + * \return true if the prediction successed, otherwise false + */ + virtual bool Predict(const cv::Mat& img, DetectionResult* result); + + /** \brief Predict the detection results for a batch of input images + * + * \param[in] imgs, The input image list, each element comes from cv::imread() + * \param[in] results The output detection result list + * \return true if the prediction successed, otherwise false + */ + virtual bool BatchPredict(const std::vector& imgs, + std::vector* results); + + /// Get preprocessor reference of YOLOv8 + virtual YOLOv8Preprocessor& GetPreprocessor() { + return preprocessor_; + } + + /// Get postprocessor reference of YOLOv8 + virtual YOLOv8Postprocessor& GetPostprocessor() { + return postprocessor_; + } + + protected: + bool Initialize(); + YOLOv8Preprocessor preprocessor_; + YOLOv8Postprocessor postprocessor_; +}; + +} // namespace detection +} // namespace vision +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/contrib/yolov8/yolov8_pybind.cc b/fastdeploy/vision/detection/contrib/yolov8/yolov8_pybind.cc new file mode 100644 index 0000000000..16c95bdb3c --- /dev/null +++ b/fastdeploy/vision/detection/contrib/yolov8/yolov8_pybind.cc @@ -0,0 +1,122 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/pybind/main.h" + +namespace fastdeploy { +void BindYOLOv8(pybind11::module& m) { + pybind11::class_(m, + "YOLOv8Preprocessor") + .def(pybind11::init<>()) + .def( + "run", + [](vision::detection::YOLOv8Preprocessor& self, + std::vector& im_list) { + std::vector images; + for (size_t i = 0; i < im_list.size(); ++i) { + images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i]))); + } + std::vector outputs; + std::vector>> ims_info; + if (!self.Run(&images, &outputs, &ims_info)) { + throw std::runtime_error( + "Failed to preprocess the input data in YOLOv8Preprocessor."); + } + for (size_t i = 0; i < outputs.size(); ++i) { + outputs[i].StopSharing(); + } + return make_pair(outputs, ims_info); + }) + .def_property("size", &vision::detection::YOLOv8Preprocessor::GetSize, + &vision::detection::YOLOv8Preprocessor::SetSize) + .def_property("padding_value", + &vision::detection::YOLOv8Preprocessor::GetPaddingValue, + &vision::detection::YOLOv8Preprocessor::SetPaddingValue) + .def_property("is_scale_up", + &vision::detection::YOLOv8Preprocessor::GetScaleUp, + &vision::detection::YOLOv8Preprocessor::SetScaleUp) + .def_property("is_mini_pad", + &vision::detection::YOLOv8Preprocessor::GetMiniPad, + &vision::detection::YOLOv8Preprocessor::SetMiniPad) + .def_property("stride", &vision::detection::YOLOv8Preprocessor::GetStride, + &vision::detection::YOLOv8Preprocessor::SetStride); + + pybind11::class_( + m, "YOLOv8Postprocessor") + .def(pybind11::init<>()) + .def("run", + [](vision::detection::YOLOv8Postprocessor& self, + std::vector& inputs, + const std::vector>>& + ims_info) { + std::vector results; + if (!self.Run(inputs, &results, ims_info)) { + throw std::runtime_error( + "Failed to postprocess the runtime result in " + "YOLOv8Postprocessor."); + } + return results; + }) + .def("run", + [](vision::detection::YOLOv8Postprocessor& self, + std::vector& input_array, + const std::vector>>& + ims_info) { + std::vector results; + std::vector inputs; + PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true); + if (!self.Run(inputs, &results, ims_info)) { + throw std::runtime_error( + "Failed to postprocess the runtime result in " + "YOLOv8Postprocessor."); + } + return results; + }) + .def_property("conf_threshold", + &vision::detection::YOLOv8Postprocessor::GetConfThreshold, + &vision::detection::YOLOv8Postprocessor::SetConfThreshold) + .def_property("nms_threshold", + &vision::detection::YOLOv8Postprocessor::GetNMSThreshold, + &vision::detection::YOLOv8Postprocessor::SetNMSThreshold) + .def_property("multi_label", + &vision::detection::YOLOv8Postprocessor::GetMultiLabel, + &vision::detection::YOLOv8Postprocessor::SetMultiLabel); + + pybind11::class_(m, "YOLOv8") + .def(pybind11::init()) + .def("predict", + [](vision::detection::YOLOv8& self, pybind11::array& data) { + auto mat = PyArrayToCvMat(data); + vision::DetectionResult res; + self.Predict(mat, &res); + return res; + }) + .def("batch_predict", + [](vision::detection::YOLOv8& self, + std::vector& data) { + std::vector images; + for (size_t i = 0; i < data.size(); ++i) { + images.push_back(PyArrayToCvMat(data[i])); + } + std::vector results; + self.BatchPredict(images, &results); + return results; + }) + .def_property_readonly("preprocessor", + &vision::detection::YOLOv8::GetPreprocessor) + .def_property_readonly("postprocessor", + &vision::detection::YOLOv8::GetPostprocessor); +} +} // namespace fastdeploy diff --git a/fastdeploy/vision/detection/detection_pybind.cc b/fastdeploy/vision/detection/detection_pybind.cc index b46f229ae9..7e3d2b9465 100755 --- a/fastdeploy/vision/detection/detection_pybind.cc +++ b/fastdeploy/vision/detection/detection_pybind.cc @@ -29,6 +29,7 @@ void BindNanoDetPlus(pybind11::module& m); void BindPPDet(pybind11::module& m); void BindYOLOv7End2EndTRT(pybind11::module& m); void BindYOLOv7End2EndORT(pybind11::module& m); +void BindYOLOv8(pybind11::module& m); void BindRKYOLO(pybind11::module& m); void BindDetection(pybind11::module& m) { @@ -47,6 +48,7 @@ void BindDetection(pybind11::module& m) { BindNanoDetPlus(detection_module); BindYOLOv7End2EndTRT(detection_module); BindYOLOv7End2EndORT(detection_module); + BindYOLOv8(detection_module); BindRKYOLO(detection_module); } } // namespace fastdeploy diff --git a/python/fastdeploy/vision/detection/__init__.py b/python/fastdeploy/vision/detection/__init__.py index cfa19bfb7d..9b4415b35b 100755 --- a/python/fastdeploy/vision/detection/__init__.py +++ b/python/fastdeploy/vision/detection/__init__.py @@ -25,5 +25,6 @@ from .contrib.yolov6 import YOLOv6 from .contrib.yolov7end2end_trt import YOLOv7End2EndTRT from .contrib.yolov7end2end_ort import YOLOv7End2EndORT +from .contrib.yolov8 import * from .ppdet import * from .contrib.rkyolo import * diff --git a/python/fastdeploy/vision/detection/contrib/yolov5seg.py b/python/fastdeploy/vision/detection/contrib/yolov5seg.py old mode 100644 new mode 100755 index a7c35bf689..176b461df3 --- a/python/fastdeploy/vision/detection/contrib/yolov5seg.py +++ b/python/fastdeploy/vision/detection/contrib/yolov5seg.py @@ -180,7 +180,7 @@ def __init__(self, model_file, params_file, self._runtime_option, model_format) assert self.initialized, "YOLOv5Seg initialize failed." - def predict(self, input_image, conf_threshold=0.25, nms_iou_threshold=0.5): + def predict(self, input_image): """Detect an input image :param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format @@ -189,8 +189,6 @@ def predict(self, input_image, conf_threshold=0.25, nms_iou_threshold=0.5): :return: DetectionResult """ - self.postprocessor.conf_threshold = conf_threshold - self.postprocessor.nms_threshold = nms_iou_threshold return self._model.predict(input_image) def batch_predict(self, images): diff --git a/python/fastdeploy/vision/detection/contrib/yolov8.py b/python/fastdeploy/vision/detection/contrib/yolov8.py new file mode 100755 index 0000000000..5dde667952 --- /dev/null +++ b/python/fastdeploy/vision/detection/contrib/yolov8.py @@ -0,0 +1,217 @@ +# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import absolute_import +import logging +from .... import FastDeployModel, ModelFormat +from .... import c_lib_wrap as C + + +class YOLOv8Preprocessor: + def __init__(self): + """Create a preprocessor for YOLOv8 + """ + self._preprocessor = C.vision.detection.YOLOv8Preprocessor() + + def run(self, input_ims): + """Preprocess input images for YOLOv8 + + :param: input_ims: (list of numpy.ndarray)The input image + :return: list of FDTensor + """ + return self._preprocessor.run(input_ims) + + @property + def size(self): + """ + Argument for image preprocessing step, the preprocess image size, tuple of (width, height), default size = [640, 640] + """ + return self._preprocessor.size + + @property + def padding_value(self): + """ + padding value for preprocessing, default [114.0, 114.0, 114.0] + """ + # padding value, size should be the same as channels + return self._preprocessor.padding_value + + @property + def is_scale_up(self): + """ + is_scale_up for preprocessing, the input image only can be zoom out, the maximum resize scale cannot exceed 1.0, default true + """ + return self._preprocessor.is_scale_up + + @property + def is_mini_pad(self): + """ + is_mini_pad for preprocessing, pad to the minimum rectange which height and width is times of stride, default false + """ + return self._preprocessor.is_mini_pad + + @property + def stride(self): + """ + stride for preprocessing, only for mini_pad mode, default 32 + """ + return self._preprocessor.stride + + @size.setter + def size(self, wh): + assert isinstance(wh, (list, tuple)),\ + "The value to set `size` must be type of tuple or list." + assert len(wh) == 2,\ + "The value to set `size` must contatins 2 elements means [width, height], but now it contains {} elements.".format( + len(wh)) + self._preprocessor.size = wh + + @padding_value.setter + def padding_value(self, value): + assert isinstance( + value, + list), "The value to set `padding_value` must be type of list." + self._preprocessor.padding_value = value + + @is_scale_up.setter + def is_scale_up(self, value): + assert isinstance( + value, + bool), "The value to set `is_scale_up` must be type of bool." + self._preprocessor.is_scale_up = value + + @is_mini_pad.setter + def is_mini_pad(self, value): + assert isinstance( + value, + bool), "The value to set `is_mini_pad` must be type of bool." + self._preprocessor.is_mini_pad = value + + @stride.setter + def stride(self, value): + assert isinstance( + stride, int), "The value to set `stride` must be type of int." + self._preprocessor.stride = value + + +class YOLOv8Postprocessor: + def __init__(self): + """Create a postprocessor for YOLOv8 + """ + self._postprocessor = C.vision.detection.YOLOv8Postprocessor() + + def run(self, runtime_results, ims_info): + """Postprocess the runtime results for YOLOv8 + + :param: runtime_results: (list of FDTensor)The output FDTensor results from runtime + :param: ims_info: (list of dict)Record input_shape and output_shape + :return: list of DetectionResult(If the runtime_results is predict by batched samples, the length of this list equals to the batch size) + """ + return self._postprocessor.run(runtime_results, ims_info) + + @property + def conf_threshold(self): + """ + confidence threshold for postprocessing, default is 0.25 + """ + return self._postprocessor.conf_threshold + + @property + def nms_threshold(self): + """ + nms threshold for postprocessing, default is 0.5 + """ + return self._postprocessor.nms_threshold + + @property + def multi_label(self): + """ + multi_label for postprocessing, set true for eval, default is True + """ + return self._postprocessor.multi_label + + @conf_threshold.setter + def conf_threshold(self, conf_threshold): + assert isinstance(conf_threshold, float),\ + "The value to set `conf_threshold` must be type of float." + self._postprocessor.conf_threshold = conf_threshold + + @nms_threshold.setter + def nms_threshold(self, nms_threshold): + assert isinstance(nms_threshold, float),\ + "The value to set `nms_threshold` must be type of float." + self._postprocessor.nms_threshold = nms_threshold + + @multi_label.setter + def multi_label(self, value): + assert isinstance( + value, + bool), "The value to set `multi_label` must be type of bool." + self._postprocessor.multi_label = value + + +class YOLOv8(FastDeployModel): + def __init__(self, + model_file, + params_file="", + runtime_option=None, + model_format=ModelFormat.ONNX): + """Load a YOLOv8 model exported by YOLOv8. + + :param model_file: (str)Path of model file, e.g ./yolov8s.onnx + :param params_file: (str)Path of parameters file, e.g yolox/model.pdiparams, if the model_fomat is ModelFormat.ONNX, this param will be ignored, can be set as empty string + :param runtime_option: (fastdeploy.RuntimeOption)RuntimeOption for inference this model, if it's None, will use the default backend on CPU + :param model_format: (fastdeploy.ModelForamt)Model format of the loaded model + """ + super(YOLOv8, self).__init__(runtime_option) + + self._model = C.vision.detection.YOLOv8( + model_file, params_file, self._runtime_option, model_format) + assert self.initialized, "YOLOv8 initialize failed." + + def predict(self, input_image): + """Detect an input image + + :param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format + :param conf_threshold: confidence threshold for postprocessing, default is 0.25 + :param nms_iou_threshold: iou threshold for NMS, default is 0.5 + :return: DetectionResult + """ + + return self._model.predict(input_image) + + def batch_predict(self, images): + """Classify a batch of input image + + :param im: (list of numpy.ndarray) The input image list, each element is a 3-D array with layout HWC, BGR format + :return list of DetectionResult + """ + + return self._model.batch_predict(images) + + @property + def preprocessor(self): + """Get YOLOv8Preprocessor object of the loaded model + + :return YOLOv8Preprocessor + """ + return self._model.preprocessor + + @property + def postprocessor(self): + """Get YOLOv8Postprocessor object of the loaded model + + :return YOLOv8Postprocessor + """ + return self._model.postprocessor