Ffmpeg With OpenCVForUnity Example

Overview

• Integrate FFmpeg for Unity with OpenCV for Unity.
• FFmpeg receives video from various supported input sources (such as RTSP, HTTP, HLS, or local files), converts it to OpenCV's Mat class, and performs image processing.

Environment

• (Optional) ATOM Cam Swing + atomcam_tools

  can be replaced with any RTSP-compatible camera

• Windows / macOS / Linux / Android / iOS
• Unity 2021.3.35f1+
• Scripting Backend: Mono / IL2CPP
• OpenCV for Unity 3.0.0+
• FFmpeg for Unity 2.11.1+
• Runtime Inspector & Hierarchy 1.7.0+
• In-game Debug Console 1.5.9+

Setup

1. Download the latest release unitypackage from FfmpegWithOpenCVForUnityExample.unitypackage.
2. Create a new project. (ex. FfmpegWithOpenCVForUnityExample)
3. Import and Setup OpenCV for Unity.
   • Download Dnn model files by Example Assets Downloader window.
     
   • Move the files from the Assets/OpenCVForUnity/StreamingAssets folder to the Assets/StreamingAssets folder.
     
4. Import and Setup FFmpeg for Unity.
5. Import Runtime Inspector & Hierarchy.
6. Import In-game Debug Console.
7. Import FfmpegWithOpenCVForUnityExample.unitypackage.
8. Set the URL of RTSPServer you wish to receive.
   
9. Add all of the ***.unity files in the FfmpegWithOpenCVForUnityExample folder to Build Settings → Scenes In Build.
10. Build and Deploy.
    

ScreenShot