Skip to content

Commit cfc0203

Browse files
authored
[pico] openvino infer (PaddlePaddle#5687)
1 parent c70879d commit cfc0203

File tree

4 files changed

+300
-15
lines changed

4 files changed

+300
-15
lines changed
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# PicoDet OpenVINO Benchmark Demo
22

3-
本文件夹提供利用[Intel's OpenVINO Toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html)进行PicoDet测速的Benchmark Demo
3+
本文件夹提供利用[Intel's OpenVINO Toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html)进行PicoDet测速的Benchmark Demo与带后处理的模型Inference Demo
44

55
## 安装 OpenVINO Toolkit
66

@@ -13,9 +13,9 @@ pip install openvino==2022.1.0
1313

1414
详细安装步骤,可参考[OpenVINO官网](https://docs.openvinotoolkit.org/latest/get_started_guides.html)
1515

16-
## 测试
16+
## Benchmark测试
1717

18-
- 准备测试模型:根据[PicoDet](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet)中【导出及转换模型】步骤,采用不包含后处理的方式导出模型(`-o export.benchmark=True` ),并生成待测试模型简化后的onnx模型(可在下文链接中可直接下载)。同时在本目录下新建```out_onnxsim```文件夹,将导出的onnx模型放在该目录下。
18+
- 准备测试模型:根据[PicoDet](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet)中【导出及转换模型】步骤,采用不包含后处理的方式导出模型(`-o export.benchmark=True` ),并生成待测试模型简化后的onnx模型(可在下文链接中直接下载)。同时在本目录下新建```out_onnxsim```文件夹,将导出的onnx模型放在该目录下。
1919

2020
- 准备测试所用图片:本demo默认利用PaddleDetection/demo/[000000014439.jpg](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/demo/000000014439.jpg)
2121

@@ -30,28 +30,46 @@ python openvino_benchmark.py --img_path ..\..\..\..\demo\000000014439.jpg --onnx
3030
```
3131
- 注意:```--in_shape```为对应模型输入size,默认为320
3232

33-
### Inference images
33+
### Inference images(w/o 后处理)
3434

3535
```shell
3636
# Linux
3737
python openvino_benchmark.py --benchmark 0 --img_path ../../../../demo/000000014439.jpg --onnx_path out_onnxsim/picodet_s_320_coco_lcnet.onnx --in_shape 320
3838
# Windows
3939
python openvino_benchmark.py --benchmark 0 --img_path ..\..\..\..\demo\000000014439.jpg --onnx_path out_onnxsim\picodet_s_320_coco_lcnet.onnx --in_shape 320
4040
```
41+
42+
## Inference images(w/ 后处理, w/o NMS)
43+
44+
- 准备测试模型:根据[PicoDet](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet)中【导出及转换模型】步骤,采用**包含后处理****不包含NMS**的方式导出模型(`-o export.benchmark=False export.nms=False` ),并生成待测试模型简化后的onnx模型(可在下文链接中直接下载)。同时在本目录下新建```out_onnxsim_infer```文件夹,将导出的onnx模型放在该目录下。
45+
46+
- 准备测试所用图片:默认利用../../demo_onnxruntime/imgs/[bus.jpg](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/third_engine/demo_onnxruntime/imgs/bus.jpg)
47+
48+
```shell
49+
# Linux
50+
python openvino_infer.py --img_path ../../demo_onnxruntime/imgs/bus.jpg --onnx_path out_onnxsim_infer/picodet_s_320_postproccesed_woNMS.onnx --in_shape 320
51+
# Windows
52+
python openvino_infer.py --img_path ..\..\demo_onnxruntime\imgs\bus.jpg --onnx_path out_onnxsim_infer\picodet_s_320_postproccesed_woNMS.onnx --in_shape 320
53+
```
54+
- 结果:
55+
<div align="center">
56+
<img src="../../../../docs/images/res.jpg" height="500px" >
57+
</div>
58+
4159
## 结果
4260

43-
测试结果如下
61+
- 测速结果如下
4462

4563
| 模型 | 输入尺寸 | ONNX | 预测时延<sup><small>[CPU](#latency)|
4664
| :-------- | :--------: | :---------------------: | :----------------: |
47-
| PicoDet-XS | 320*320 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_320_coco_lcnet.onnx) | 3.9ms |
48-
| PicoDet-XS | 416*416 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_416_coco_lcnet.onnx) | 6.1ms |
49-
| PicoDet-S | 320*320 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_320_coco_lcnet.onnx) | 4.8ms |
50-
| PicoDet-S | 416*416 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_416_coco_lcnet.onnx) | 6.6ms |
51-
| PicoDet-M | 320*320 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_320_coco_lcnet.onnx) | 8.2ms |
52-
| PicoDet-M | 416*416 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_416_coco_lcnet.onnx) | 12.7ms |
53-
| PicoDet-L | 320*320 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_320_coco_lcnet.onnx) | 11.5ms |
54-
| PicoDet-L | 416*416 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_416_coco_lcnet.onnx) | 20.7ms |
55-
| PicoDet-L | 640*640 | [model](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_640_coco.onnx) | 62.5ms |
65+
| PicoDet-XS | 320*320 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_320_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_320_coco_lcnet.onnx) | 3.9ms |
66+
| PicoDet-XS | 416*416 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_416_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_xs_416_coco_lcnet.onnx) | 6.1ms |
67+
| PicoDet-S | 320*320 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_320_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_320_coco_lcnet.onnx) | 4.8ms |
68+
| PicoDet-S | 416*416 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_416_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_s_416_coco_lcnet.onnx) | 6.6ms |
69+
| PicoDet-M | 320*320 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_320_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_320_coco_lcnet.onnx) | 8.2ms |
70+
| PicoDet-M | 416*416 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_416_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_m_416_coco_lcnet.onnx) | 12.7ms |
71+
| PicoDet-L | 320*320 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_320_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_320_coco_lcnet.onnx) | 11.5ms |
72+
| PicoDet-L | 416*416 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_416_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_416_coco_lcnet.onnx) | 20.7ms |
73+
| PicoDet-L | 640*640 | [( w/ 后处理;w/o NMS)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_640_lcnet_postproccesed_woNMS.onnx) &#124; [( w/o 后处理)](https://paddledet.bj.bcebos.com/deploy/third_engine/picodet_l_640_coco_lcnet.onnx) | 62.5ms |
5674

5775
- <a name="latency">测试环境:</a> 英特尔酷睿i7 10750H CPU。

deploy/third_engine/demo_openvino/python/openvino_benchmark.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -339,7 +339,7 @@ def benchmark(test_image, compiled_model):
339339
parser.add_argument(
340340
'--img_path',
341341
type=str,
342-
default='demo/000000014439.jpg',
342+
default='../../../../demo/000000014439.jpg',
343343
help="image path")
344344
parser.add_argument(
345345
'--onnx_path',
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,267 @@
1+
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import cv2
16+
import numpy as np
17+
import argparse
18+
from scipy.special import softmax
19+
from openvino.runtime import Core
20+
21+
22+
def image_preprocess(img_path, re_shape):
23+
img = cv2.imread(img_path)
24+
img = cv2.resize(
25+
img, (re_shape, re_shape), interpolation=cv2.INTER_LANCZOS4)
26+
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
27+
img = np.transpose(img, [2, 0, 1]) / 255
28+
img = np.expand_dims(img, 0)
29+
img_mean = np.array([0.485, 0.456, 0.406]).reshape((3, 1, 1))
30+
img_std = np.array([0.229, 0.224, 0.225]).reshape((3, 1, 1))
31+
img -= img_mean
32+
img /= img_std
33+
return img.astype(np.float32)
34+
35+
36+
def get_color_map_list(num_classes):
37+
color_map = num_classes * [0, 0, 0]
38+
for i in range(0, num_classes):
39+
j = 0
40+
lab = i
41+
while lab:
42+
color_map[i * 3] |= (((lab >> 0) & 1) << (7 - j))
43+
color_map[i * 3 + 1] |= (((lab >> 1) & 1) << (7 - j))
44+
color_map[i * 3 + 2] |= (((lab >> 2) & 1) << (7 - j))
45+
j += 1
46+
lab >>= 3
47+
color_map = [color_map[i:i + 3] for i in range(0, len(color_map), 3)]
48+
return color_map
49+
50+
51+
def draw_box(srcimg, results, class_label):
52+
label_list = list(
53+
map(lambda x: x.strip(), open(class_label, 'r').readlines()))
54+
for i in range(len(results)):
55+
color_list = get_color_map_list(len(label_list))
56+
clsid2color = {}
57+
classid, conf = int(results[i, 0]), results[i, 1]
58+
xmin, ymin, xmax, ymax = int(results[i, 2]), int(results[i, 3]), int(
59+
results[i, 4]), int(results[i, 5])
60+
61+
if classid not in clsid2color:
62+
clsid2color[classid] = color_list[classid]
63+
color = tuple(clsid2color[classid])
64+
65+
cv2.rectangle(srcimg, (xmin, ymin), (xmax, ymax), color, thickness=2)
66+
print(label_list[classid] + ': ' + str(round(conf, 3)))
67+
cv2.putText(
68+
srcimg,
69+
label_list[classid] + ':' + str(round(conf, 3)), (xmin, ymin - 10),
70+
cv2.FONT_HERSHEY_SIMPLEX,
71+
0.8, (0, 255, 0),
72+
thickness=2)
73+
return srcimg
74+
75+
76+
def hard_nms(box_scores, iou_threshold, top_k=-1, candidate_size=200):
77+
"""
78+
Args:
79+
box_scores (N, 5): boxes in corner-form and probabilities.
80+
iou_threshold: intersection over union threshold.
81+
top_k: keep top_k results. If k <= 0, keep all the results.
82+
candidate_size: only consider the candidates with the highest scores.
83+
Returns:
84+
picked: a list of indexes of the kept boxes
85+
"""
86+
scores = box_scores[:, -1]
87+
boxes = box_scores[:, :-1]
88+
picked = []
89+
indexes = np.argsort(scores)
90+
indexes = indexes[-candidate_size:]
91+
while len(indexes) > 0:
92+
current = indexes[-1]
93+
picked.append(current)
94+
if 0 < top_k == len(picked) or len(indexes) == 1:
95+
break
96+
current_box = boxes[current, :]
97+
indexes = indexes[:-1]
98+
rest_boxes = boxes[indexes, :]
99+
iou = iou_of(
100+
rest_boxes,
101+
np.expand_dims(
102+
current_box, axis=0), )
103+
indexes = indexes[iou <= iou_threshold]
104+
105+
return box_scores[picked, :]
106+
107+
108+
def iou_of(boxes0, boxes1, eps=1e-5):
109+
"""Return intersection-over-union (Jaccard index) of boxes.
110+
Args:
111+
boxes0 (N, 4): ground truth boxes.
112+
boxes1 (N or 1, 4): predicted boxes.
113+
eps: a small number to avoid 0 as denominator.
114+
Returns:
115+
iou (N): IoU values.
116+
"""
117+
overlap_left_top = np.maximum(boxes0[..., :2], boxes1[..., :2])
118+
overlap_right_bottom = np.minimum(boxes0[..., 2:], boxes1[..., 2:])
119+
120+
overlap_area = area_of(overlap_left_top, overlap_right_bottom)
121+
area0 = area_of(boxes0[..., :2], boxes0[..., 2:])
122+
area1 = area_of(boxes1[..., :2], boxes1[..., 2:])
123+
return overlap_area / (area0 + area1 - overlap_area + eps)
124+
125+
126+
def area_of(left_top, right_bottom):
127+
"""Compute the areas of rectangles given two corners.
128+
Args:
129+
left_top (N, 2): left top corner.
130+
right_bottom (N, 2): right bottom corner.
131+
Returns:
132+
area (N): return the area.
133+
"""
134+
hw = np.clip(right_bottom - left_top, 0.0, None)
135+
return hw[..., 0] * hw[..., 1]
136+
137+
138+
class PicoDetNMS(object):
139+
"""
140+
Args:
141+
input_shape (int): network input image size
142+
scale_factor (float): scale factor of ori image
143+
"""
144+
145+
def __init__(self,
146+
input_shape,
147+
scale_x,
148+
scale_y,
149+
strides=[8, 16, 32, 64],
150+
score_threshold=0.4,
151+
nms_threshold=0.5,
152+
nms_top_k=1000,
153+
keep_top_k=100):
154+
self.input_shape = input_shape
155+
self.scale_x = scale_x
156+
self.scale_y = scale_y
157+
self.strides = strides
158+
self.score_threshold = score_threshold
159+
self.nms_threshold = nms_threshold
160+
self.nms_top_k = nms_top_k
161+
self.keep_top_k = keep_top_k
162+
163+
def __call__(self, decode_boxes, select_scores):
164+
batch_size = 1
165+
out_boxes_list = []
166+
for batch_id in range(batch_size):
167+
# nms
168+
bboxes = np.concatenate(decode_boxes, axis=0)
169+
confidences = np.concatenate(select_scores, axis=0)
170+
picked_box_probs = []
171+
picked_labels = []
172+
for class_index in range(0, confidences.shape[1]):
173+
probs = confidences[:, class_index]
174+
mask = probs > self.score_threshold
175+
probs = probs[mask]
176+
if probs.shape[0] == 0:
177+
continue
178+
subset_boxes = bboxes[mask, :]
179+
box_probs = np.concatenate(
180+
[subset_boxes, probs.reshape(-1, 1)], axis=1)
181+
box_probs = hard_nms(
182+
box_probs,
183+
iou_threshold=self.nms_threshold,
184+
top_k=self.keep_top_k, )
185+
picked_box_probs.append(box_probs)
186+
picked_labels.extend([class_index] * box_probs.shape[0])
187+
188+
if len(picked_box_probs) == 0:
189+
out_boxes_list.append(np.empty((0, 4)))
190+
191+
else:
192+
picked_box_probs = np.concatenate(picked_box_probs)
193+
194+
# resize output boxes
195+
picked_box_probs[:, 0] *= self.scale_x
196+
picked_box_probs[:, 2] *= self.scale_x
197+
picked_box_probs[:, 1] *= self.scale_y
198+
picked_box_probs[:, 3] *= self.scale_y
199+
200+
# clas score box
201+
out_boxes_list.append(
202+
np.concatenate(
203+
[
204+
np.expand_dims(
205+
np.array(picked_labels),
206+
axis=-1), np.expand_dims(
207+
picked_box_probs[:, 4], axis=-1),
208+
picked_box_probs[:, :4]
209+
],
210+
axis=1))
211+
212+
out_boxes_list = np.concatenate(out_boxes_list, axis=0)
213+
return out_boxes_list
214+
215+
216+
def detect(img_file, compiled_model, class_label):
217+
output = compiled_model.infer_new_request({0: test_image})
218+
result_ie = list(output.values())
219+
220+
decode_boxes = []
221+
select_scores = []
222+
num_outs = int(len(result_ie) / 2)
223+
for out_idx in range(num_outs):
224+
decode_boxes.append(result_ie[out_idx])
225+
select_scores.append(result_ie[out_idx + num_outs])
226+
227+
image = cv2.imread(img_file, 1)
228+
scale_x = image.shape[1] / test_image.shape[3]
229+
scale_y = image.shape[0] / test_image.shape[2]
230+
231+
nms = PicoDetNMS(test_image.shape[2:], scale_x, scale_y)
232+
np_boxes = nms(decode_boxes, select_scores)
233+
234+
res_image = draw_box(image, np_boxes, class_label)
235+
236+
cv2.imwrite('res.jpg', res_image)
237+
cv2.imshow("res", res_image)
238+
cv2.waitKey()
239+
240+
241+
if __name__ == '__main__':
242+
243+
parser = argparse.ArgumentParser()
244+
parser.add_argument(
245+
'--img_path',
246+
type=str,
247+
default='../../demo_onnxruntime/imgs/bus.jpg',
248+
help="image path")
249+
parser.add_argument(
250+
'--onnx_path',
251+
type=str,
252+
default='out_onnxsim_infer/picodet_s_320_postproccesed_woNMS.onnx',
253+
help="onnx filepath")
254+
parser.add_argument('--in_shape', type=int, default=320, help="input_size")
255+
parser.add_argument(
256+
'--class_label',
257+
type=str,
258+
default='coco_label.txt',
259+
help="class label file")
260+
args = parser.parse_args()
261+
262+
ie = Core()
263+
net = ie.read_model(args.onnx_path)
264+
test_image = image_preprocess(args.img_path, args.in_shape)
265+
compiled_model = ie.compile_model(net, 'CPU')
266+
267+
detect(args.img_path, compiled_model, args.class_label)

docs/images/res.jpg

481 KB
Loading

0 commit comments

Comments
 (0)