ONNX Runtime Inference

Introduction

(clh)ONNX Runtime C++ inference example for image classification using CPU and CUDA.

Dependencies

• CMake 3.20.1
• ONNX Runtime 1.12.0
• OpenCV 4.5.2

自己尝试成功的版本：

• CMake 3.26.0
• ONNX Runtime 1.12.1
• OpenCV 4.2.0

Usages

Build Docker Image

$ docker build -f docker/onnxruntime-cuda.Dockerfile --no-cache --tag=onnxruntime-cuda:1.12.0 .

Run Docker Container

$ docker run -it --rm --gpus device=0 -v $(pwd):/mnt onnxruntime-cuda:1.12.0

可以不用docker，从下边开始

Build Example

$ cmake -B build
$ cmake --build build --config Release --parallel

Run Example

$ cd build/src/
$ ./inference  --use_cpu
Inference Execution Provider: CPU
Number of Input Nodes: 1
Number of Output Nodes: 1
Input Name: data
Input Type: float
Input Dimensions: [1, 3, 224, 224]
Output Name: squeezenet0_flatten0_reshape0
Output Type: float
Output Dimensions: [1, 1000]
Predicted Label ID: 92
Predicted Label: n01828970 bee eater
Uncalibrated Confidence: 0.996137
Minimum Inference Latency: 7.45 ms

$ cd build/src/
$ ./inference  --use_cuda
Inference Execution Provider: CUDA
Number of Input Nodes: 1
Number of Output Nodes: 1
Input Name: data
Input Type: float
Input Dimensions: [1, 3, 224, 224]
Output Name: squeezenet0_flatten0_reshape0
Output Type: float
Output Dimensions: [1, 1000]
Predicted Label ID: 92
Predicted Label: n01828970 bee eater
Uncalibrated Confidence: 0.996137
Minimum Inference Latency: 0.98 ms

References

• ONNX Runtime C++ Inference

Models Download

脸部分析和换脸模型下载可参考facefusion项目代码face_analyser.py和face_swapper.py

• facefusion