deepstream-segmentation
Folders and files
Name | Name | Last commit date | ||
---|---|---|---|---|
parent directory.. | ||||
################################################################################ # SPDX-FileCopyrightText: Copyright (c) 2019-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ################################################################################ Prerequisites: - DeepStreamSDK 7.0 **NOT SUPPORTED BY 7.1** - Python 3.10 - Gst-python - NumPy package <2.0, >=1.22 (2.0 and above not supported) - OpenCV package To install required packages: $ sudo apt update $ sudo apt install python3-numpy python3-opencv -y If on Jetson, the libgomp.so.1 must be added to LD_PRELOAD: $ export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1 To run: $ python3 deepstream_segmentation.py <config_file> <jpeg/mjpeg stream> <FOLDER NAME TO SAVE FRAMES> This document shall describe the sample deepstream-segmentation application. It is meant for simple demonstration of how to use the various DeepStream SDK elements in the pipeline and extract meaningful insights from a video stream such as segmentation masks and respective color mapping for segmentation visualization. This sample creates instance of "nvinfer" element. Instance of the "nvinfer" uses TensorRT API to execute inferencing on a model. Using a correct configuration for a nvinfer element instance is therefore very important as considerable behaviors of the instance are parameterized through these configs. For reference, here are the config files used for this sample : 1. The 4-class segmentation model configured through dstest_segmentation_config_semantic.txt 2. The 2-class segmentation model configured through dstest_segmentation_config_industrial.txt In this sample, we first create one instance of "nvinfer", referred as the pgie. for semantic segmentation, it needs semantic model which can get 4 classes map, including backgroud, car, person, bicycle. Then "nvsegvidsual" plugin chooses 4 different colors for them and to display. Similarlty for industrial segmentation, it needs industrial model which can only get defective area map. Later nvinfer element attach some MetaData to the buffer. By attaching the probe function at the end of the pipeline, one can extract meaningful information from this inference. Please refer the "tiler_src_pad_buffer_probe" function in the sample code. For details on the Metadata format, refer to the file "gstnvdsmeta.h". In this probe we demonstrate extracting the masks and color mapping for segmentation visualization using opencv and numpy.