Skip to content

Latest commit

 

History

History
123 lines (83 loc) · 6.96 KB

README.md

File metadata and controls

123 lines (83 loc) · 6.96 KB
Raw

YOLO-Streaming

Hi, this repository documents the process of pushing streams on some ultra-lightweight nets. The general steps are that opencv calls the board（like Raspberry Pi）'s camera, transmits the detected live video to an ultra-lightweight network like yolo-fastest, YOLOv4-tiny, YOLOv5s-onnx, and then talks about pushing the processed video frames to the web using the flask lightweight framework, which basically guarantees real-time performance.

step

2021-08-01 Update

Add YOLOX and Test performance

Requirements

Please install the following packages first（for dnn）

  • Linux & MacOS & window
  • python>= 3.6.0
  • opencv-python>= 4.2.X
  • flask>= 1.0.0

Please install the following packages first（for ncnn）

  • Linux & MacOS & window
  • Visual Studio 2019
  • cmake-3.16.5
  • protobuf-3.4.0
  • opencv-3.4.0
  • vulkan-1.14.8

inference

Equipment Computing backend System Framework input_size Run time
Raspberrypi 3B 4xCortex-A53 Linux(arm64) dnn 320 89ms
Intel Core i5-4210 window10（x64） dnn 320 21ms
Equipment Computing backend System Framework input_size Run time
Raspberrypi 3B 4xCortex-A53 Linux(arm64) dnn 320 315ms
Intel Core i5-4210 window10（x64） dnn 320 41ms
Equipment Computing backend System Framework input_size Run time
Raspberrypi 3B 4xCortex-A53 Linux(arm64) dnn 320 673ms
Intel Core i5-4210 window10（x64） dnn 320 131ms
Raspberrypi 3B 4xCortex-A53 Linux(arm64) ncnn 160 716ms
Intel Core i5-4210 window10（x64） ncnn 160 197ms
Equipment Computing backend System Framework input_size Run time
Raspberrypi 3B 4xCortex-A53 Linux(arm64) dnn 320 173ms
Intel Core i5-4210 window10（x64） dnn 320 33ms

updating. . .

Demo

First of all, I have tested this demo in window, mac and linux environments and it works in all of them.

Run v3_fastest.py Run v4_tiny.py Run v5_dnn.py Run vx_ort.py

  • Inference images use python xx.py

Run app.py -（Push-Streaming online）

  • Inference with v3-fastest python app.py --model v3_fastest
  • Inference with v4-tiny python app.py --model v4_tiny
  • Inference with v5-dnn python app.py --model v5_dnn
  • Inference with NanoDet python app.py --model vx_ort

⚡ Please note! Be sure to be on the same LAN！

Demo Effects

Run v3_fastest.py

  • image→video→capture→push stream

stream

Run v4_tiny.py

  • image→video→capture→push stream 任需优化，后续补充量化版本，待更新...

Run v5_dnn.py

  • image(473 ms / Inference Image / Core i5-4210)→video→capture(213 ms / Inference Image / Core i5-4210)→push stream

2021-04-26 记：有趣的是，用onnx＋dnn的方式调用v5s的模型，推理图片要比摄像头处理帧多花一倍的时间，看了很久，还是找不出问题所在，希望看到的大佬可以帮看看代码，点破问题所在，感谢！

2021-05-01 更：今天找到了问题所在，因为v5_dnn.py文件中有个推理时间画在帧图上的功能（cv2.putText(frame, "TIME: " + str(localtime), (8, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 2)），而这个功能居然花费了每帧推理后处理时间的2/3（大约一帧是50-80ms），后续的版本全部去掉，改成终端显示，每帧推理时间由190ms→130ms，也是恐怖额。

Supplement

This is a DNN repository that integrates the current detection algorithms. You may ask why call the model with DNN, not just git clone the whole framework down? In fact, when we are working with models, it is more advisable to separate training and inference. More, when you deploy models on a customer's production line, if you package up the training code and the training-dependent environment for each set of models (yet the customer's platform only needs you to infer, no training required for you), you will be dead after a few sets of models. As an example, here is the docker for the same version of yolov5 (complete code and dependencies & inference code and dependencies). The entire docker has enough memory to support about four sets of inference dockers.

这是一个整合了当前检测算法的DNN资源库。你可能会问，为什么用DNN调用模型，而不是直接用git克隆整个框架下来？事实上，当我们在处理模型的时候，把训练和推理分开是比较明智的。更多的是，当你在客户的生产线上部署模型的时候，如果你把每套模型的训练代码和依赖训练的环境打包起来（然而客户的平台只需要你推理，不需要你训练），那么你在几套模型之后就凉了呀。作为一个例子，这里是同一版本的yolov5的docker（完整的代码和依赖性→6.06G &推理代码和依赖性→0.4G）。整个docker有足够的内存来支持大约15套推理docker。

Thanks

other