Parsers and Messages V2 #116

jkbmrz · 2024-10-24T10:46:15Z

This PR merges the dev branch into the main for the v0.0.3 release of the depthai-nodes library. Primarily, it updates how parsers are structured (as defined by the BaseParser) and built - manually (1) during or (2) after initialization (using setters), or (3) automatically by calling the build() method. The latter allows building the parser from a NN Archive (see the newly added ParserGenerator). Moreover, we are adding the ParsingNeuralNetwork node that wraps the NeuralNetwork node with parsing capabilities. Additionally, we have updated numerous parsers, messages and creator functions.

Overview:

parsers
- adding an abstract BaseParser that defines basic parser structure and that all parsers must inherit from.
- adding: ClassificationSequenceParser, DetectionParser, EmbeddingsParser, LaneDetectionParser, MapOutputParser, PPTextDetectionParser, RegressionParser, XFeatMonoParser, XFeatStereoParser (+ relevant utils);
- removing: AgeGenderParser, MPHandLandmarkParser, MonocularDepthParser, ThermalImageParser, XFeatParser (+ relevant utils);
messages
- adding: Cluster/s, Keypoint, Map2D, Prediction/s, SegmentationMask (+ relevant utils);
- updates:
  - ImgDetectionExtended: bbox coordinates changed from xyxy to xywh and must be relative;
  - Keypoints: keypoint coordinates must be relative;
- removing: HandKeypoints, AgeGender, SegmentationMasks (+ relevant utils);
- adding improved type hinting and type checking;
creators
- adding new ones for the newly added messages and removing the outdated;
misc
- adding ParserGenerator, a general interface for instantiating parsers from NN Archives;
- adding ParsingNeuralNetwork node that wraps the NeuralNetwork node and adds parsing capabilities;
- adding TilesPatcher and Tiling helper nodes (+ relevant utils);
- fixing pre-commit version
- fixing tests

* Add support for different input sizes. * Support extended palm detection in examples. * Pre-commit fix. * Add normalization.

…r characters. This PR adds a classification_sequence_message that returns a sequence of charecters corresponding to maximum probability in the sequence. The message has optional parameters to remove duplicates and ignore background class indexes. In addition this PR also adds a PaddleOCR Parser, which utilizes the classification_sequence message.

Co-authored-by: KlemenSkrlj <[email protected]>

* Add LaneDetectionParser. * Add tests. * Add LaneDetection parser to parsers. * Example added. * typo. * Precommit fix. * [Automated] Updated coverage badge --------- Co-authored-by: GitHub Actions <[email protected]>

* fix: ParsingNeuralNetwork build overload method names * fix: YoloExtendedParser imports * fix: remove double-calling the np.array() * fix: make sure we pass floats to the ImgDetectionExtended setters

* Add keypoint confidences to YOLO. * Refactor FastSAM. * Fix Yolo build method. * Get subtype from archive instead of yolo_version. * Typo. * Move mask related functions. * Docstring fix. * Docstring fix. * Remove comments. * Change yolo_version to subtype. * Update arguments. * Update depthai_nodes/ml/parsers/yolo.py Co-authored-by: KlemenSkrlj <[email protected]> * Docstring fix. Co-authored-by: KlemenSkrlj <[email protected]> * Docstring param description fix. Co-authored-by: KlemenSkrlj <[email protected]> * Error printing fix. Co-authored-by: KlemenSkrlj <[email protected]> --------- Co-authored-by: KlemenSkrlj <[email protected]>

* Add a simple parser for embeddings models * add setOutputLayerNames set method

* fix: update config usage to latest DAIv3 version * fix: clip scores in HRNetParser * feat: allow keypoints message creation with no conf threshold * refactor: unify docstrings * feat: remove run() method and output_layer_name parameter from the DetectionParser * refactor: unify the order of parser methods * feat: add missing setters and unify existing ones * feat: remove default values from DetectionParser * feat: move score_threshold from children to KeypointParser * feat: unify how to handle missing output names * feat: remove default output_layer_name option for HRNetParser * feat: call head_config arguments with .get() to prevent errors * updated pre-commit version * added manual end2end test script * feat: refactor messages * fix: creators unittests * feat: remove SegmentationMask message * fix: pre-commit formatting * Correct build in PalmDetection parser. * Correct build in PPTextDetection parser. * Correct build in SCRFD. * Small fix in YuNet. * Python v3.8 supported. * Python v3.8 support in helpers. * Use alpha4. * Small SCRFD fix. * fix: ImageOutputParser get outputs * fix: MapOutputParser set timestamp * feat: rename SegmentationMasks to SegmentationMask * feat: force keypoint and bbox coordinates to 0-1 * fix: PPTextDetectionParser docstring * fix: keypoint and bbox requirements * Boundaries fix. * Masks fix. * feat: modify masks setter and getter methods in ImgDetectionsExtended message --------- Co-authored-by: klemen1999 <[email protected]> Co-authored-by: kkeroo <[email protected]>

github-actions · 2024-10-24T13:24:59Z

Test Results

125 tests 125 ✅ 1s ⏱️
1 suites 0 💤
1 files 0 ❌

Results for commit 78d2914.

github-actions · 2024-10-24T13:25:02Z

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
3716	1190	32%	0%	🟢

New Files

File	Coverage	Status
depthai_nodes/ml/messages/creators/regression.py	100%	🟢
depthai_nodes/ml/messages/prediction.py	89%	🟢
depthai_nodes/ml/parsers/base_parser.py	70%	🟢
depthai_nodes/ml/parsers/classification_sequence.py	21%	🟢
depthai_nodes/ml/parsers/detection.py	29%	🟢
depthai_nodes/ml/parsers/embeddings.py	36%	🟢
depthai_nodes/ml/parsers/regression.py	29%	🟢
depthai_nodes/ml/parsers/utils/bbox_format_converters.py	8%	🟢
depthai_nodes/ml/parsers/utils/decode_head.py	25%	🟢
depthai_nodes/ml/parsers/utils/keypoints.py	44%	🟢
depthai_nodes/ml/parsers/utils/masks_utils.py	24%	🟢
depthai_nodes/ml/parsers/utils/yunet.py	15%	🟢
depthai_nodes/ml/parsers/xfeat.py	19%	🟢
depthai_nodes/parser_generator.py	0%	🟢
depthai_nodes/parsing_neural_network.py	0%	🟢
TOTAL	34%	🟢

Modified Files

File	Coverage	Status
depthai_nodes/ml/helpers/tiles_patcher.py	0%	🟢
depthai_nodes/ml/helpers/tiling.py	0%	🟢
depthai_nodes/ml/helpers/utils/nms.py	0%	🟢
depthai_nodes/ml/helpers/utils/to_planar.py	0%	🟢
depthai_nodes/ml/messages/init.py	100%	🟢
depthai_nodes/ml/messages/classification.py	84%	🟢
depthai_nodes/ml/messages/clusters.py	79%	🟢
depthai_nodes/ml/messages/creators/init.py	100%	🟢
depthai_nodes/ml/messages/creators/classification.py	98%	🟢
depthai_nodes/ml/messages/creators/detection.py	94%	🟢
depthai_nodes/ml/messages/creators/keypoints.py	100%	🟢
depthai_nodes/ml/messages/creators/segmentation.py	65%	🟢
depthai_nodes/ml/messages/img_detections.py	74%	🟢
depthai_nodes/ml/messages/keypoints.py	84%	🟢
depthai_nodes/ml/messages/lines.py	89%	🟢
depthai_nodes/ml/messages/map.py	90%	🟢
depthai_nodes/ml/messages/segmentation.py	43%	🟢
depthai_nodes/ml/parsers/init.py	100%	🟢
depthai_nodes/ml/parsers/classification.py	21%	🟢
depthai_nodes/ml/parsers/fastsam.py	16%	🟢
depthai_nodes/ml/parsers/hrnet.py	25%	🟢
depthai_nodes/ml/parsers/image_output.py	26%	🟢
depthai_nodes/ml/parsers/keypoints.py	22%	🟢
depthai_nodes/ml/parsers/lane_detection.py	21%	🟢
depthai_nodes/ml/parsers/map_output.py	24%	🟢
depthai_nodes/ml/parsers/mediapipe_palm_detection.py	16%	🟢
depthai_nodes/ml/parsers/mlsd.py	22%	🟢
depthai_nodes/ml/parsers/ppdet.py	30%	🟢
depthai_nodes/ml/parsers/scrfd.py	19%	🟢
depthai_nodes/ml/parsers/segmentation.py	20%	🟢
depthai_nodes/ml/parsers/superanimal_landmarker.py	30%	🟢
depthai_nodes/ml/parsers/utils/init.py	100%	🟢
depthai_nodes/ml/parsers/utils/fastsam.py	13%	🟢
depthai_nodes/ml/parsers/utils/nms.py	16%	🟢
depthai_nodes/ml/parsers/utils/ppdet.py	10%	🟢
depthai_nodes/ml/parsers/utils/scrfd.py	8%	🟢
depthai_nodes/ml/parsers/utils/yolo.py	22%	🟢
depthai_nodes/ml/parsers/yolo.py	20%	🟢
depthai_nodes/ml/parsers/yunet.py	15%	🟢
TOTAL	43%	🟢

updated for commit: 78d2914 by action🐍

klemen1999

LGTM, already checked when merging from parsers-v2 branch to dev

klemen1999 and others added 30 commits October 24, 2024 13:50

Update README.md

54bd0af

YOLOv8 kpts and seg example. (#48)

dbd6cd6

Extend MPPalmDetection parser. (#49)

09cfae3

* Add support for different input sizes. * Support extended palm detection in examples. * Pre-commit fix. * Add normalization.

feat: add support for metric depth

ac63da1

fix: broken depth tests

735f55b

fix: pre-commit

7ea5776

fix: change default depth limit to float

a7f3bf8

feat: add test for metric depth map

8be05e4

feat: add parser for DM-Count crowd density estimation model

2554a61

fix: pre-commit

ed11628

feat: add property methods to Map2D message

c6bea83

fix: density map dimensions

4adb77e

fix: pre-commit

9b9b539

feat: unify map parsers

6d2c36d

feat: unify map message creators

0cfb87d

refactor: simplify map message file name

444a4f6

feat: add map tests

acd43cd

Adds FastSAM to example. (#58)

ba24aa0

Added PaddlePaddle Text detection model parser.

39ab5ca

Fixes Shape Error bug

859bdee

Revision.

a88aee2

Remove syntax error

825a4f2

Co-authored-by: KlemenSkrlj <[email protected]>

Renamed some functions.

ec03a47

Lane detection parser. (#61)

085506c

* Add LaneDetectionParser. * Add tests. * Add LaneDetection parser to parsers. * Example added. * typo. * Precommit fix. * [Automated] Updated coverage badge --------- Co-authored-by: GitHub Actions <[email protected]>

[Automated] Updated coverage badge

bcd4d16

PaddleOCR Parser and Classification Sequence message

9e5560a

Added unit tests and modified some assertions

a89f9e2

Doc string fix

e0e8e34

small fix

4e34a65

jkbmrz and others added 6 commits October 24, 2024 13:52

Fix: Adding miniature fixes (#108)

3f2ac36

* fix: ParsingNeuralNetwork build overload method names * fix: YoloExtendedParser imports * fix: remove double-calling the np.array() * fix: make sure we pass floats to the ImgDetectionExtended setters

Handle special case in Segmentation parser. (#112)

52b45d3

Parser for Embeddings NN models (#114)

3071967

* Add a simple parser for embeddings models * add setOutputLayerNames set method

Normalized coordinates in text detection parser.

cd9ea6a

klemen1999 force-pushed the dev branch from e59ab15 to cd9ea6a Compare October 24, 2024 11:58

github-actions bot added messages Changes affecting ml.messages parsers Changes affecting ml.parsers examples Changes affecting examples. release New version release labels Oct 24, 2024

github-actions bot assigned jkbmrz Oct 24, 2024

klemen1999 and others added 5 commits October 24, 2024 14:03

pre-commit fix

8527cab

fixed docstrings

64d2820

pre-commit on docs

bea79e5

fix: unittests

f243e1c

fix: pre-commit

78d2914

[Automated] Updated coverage badge

d36d9b6

jkbmrz marked this pull request as ready for review October 24, 2024 13:27

jkbmrz requested review from tersekmatija, kkeroo and klemen1999 as code owners October 24, 2024 13:27

klemen1999 approved these changes Oct 24, 2024

View reviewed changes

kkeroo approved these changes Oct 24, 2024

View reviewed changes

klemen1999 merged commit b4a44e3 into main Oct 24, 2024

klemen1999 deleted the dev branch October 24, 2024 13:32

klemen1999 restored the dev branch October 24, 2024 13:32

jkbmrz deleted the dev branch October 30, 2024 11:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parsers and Messages V2 #116

Parsers and Messages V2 #116

jkbmrz commented Oct 24, 2024 •

edited

Loading

github-actions bot commented Oct 24, 2024

github-actions bot commented Oct 24, 2024

klemen1999 left a comment

Parsers and Messages V2 #116

Parsers and Messages V2 #116

Conversation

jkbmrz commented Oct 24, 2024 • edited Loading

github-actions bot commented Oct 24, 2024

Test Results

github-actions bot commented Oct 24, 2024

☂️ Python Coverage

Overall Coverage

New Files

Modified Files

klemen1999 left a comment

Choose a reason for hiding this comment

jkbmrz commented Oct 24, 2024 •

edited

Loading