Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic Labeling #203

Draft
wants to merge 40 commits into
base: ros2-devel
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8ed6683
init
sriramk117 Aug 7, 2024
dcb74b7
added neccessary callback functions
sriramk117 Aug 11, 2024
c611e78
implemented functionality to run sam and groundingdino
sriramk117 Aug 13, 2024
4fb0258
wrote vision pipeline and execute callback
sriramk117 Aug 14, 2024
5d81216
created result message returned by vision pipeline
sriramk117 Aug 14, 2024
bfcaeaa
modified launch file and created yaml file for parameters
sriramk117 Aug 14, 2024
19e9275
updated setup.py and modified parameters
sriramk117 Aug 16, 2024
c29520d
Merge branch 'ros2-devel' into sriramk/semantic-labeling
sriramk117 Aug 16, 2024
4e7391f
added requirements to install and fixed imports
sriramk117 Aug 16, 2024
9c43fc4
changed grounding dino path and added checkpoint
sriramk117 Sep 12, 2024
31551e4
Added config file + fixed image transformations
sriramk117 Sep 12, 2024
3ec7b50
Added GroundingDINO visualization function
sriramk117 Sep 14, 2024
9dc9a40
created GroundingDINO publisher for testing
sriramk117 Sep 16, 2024
929e570
added more testing code for bbox visualization
sriramk117 Sep 16, 2024
e1ebf8b
fixed groundingdino results visualization
sriramk117 Sep 18, 2024
704caa1
corrected image preprocessing?
sriramk117 Sep 19, 2024
4f9305d
groundingdino works!
sriramk117 Sep 23, 2024
024c71c
masks are now displayable
sriramk117 Sep 23, 2024
c78cd4a
record vision pipeline inference time
sriramk117 Sep 24, 2024
e503800
wrote code to generate mask messages during action calls
sriramk117 Sep 27, 2024
648a46e
masks msgs are generated but action keeps aborting
sriramk117 Sep 30, 2024
3032f65
Added gpt-4o query functionality
sriramk117 Nov 7, 2024
85f9577
groundingdino can be downloaded via github url
Nov 8, 2024
0049598
updated comments/code quality changes
Nov 8, 2024
e9fd4d5
invoking gpt-4o has been transformed into a service
sriramk117 Nov 8, 2024
9d52d98
segment all items action now takes a single string as input
sriramk117 Nov 8, 2024
30bc036
added env variables
sriramk117 Nov 9, 2024
4d3b27c
environment variables not loading?
sriramk117 Nov 9, 2024
94af48e
ran black formatter
sriramk117 Nov 9, 2024
29ed345
Merge branch 'ros2-devel' into sriramk/semantic-labeling
sriramk117 Nov 9, 2024
23577ae
changes to segmentallitems node initializing it as a perception node
sriramk117 Nov 9, 2024
3688541
fixed error of topics not being received by segmentallitems action
sriramk117 Nov 9, 2024
195b123
code cleanup
Nov 9, 2024
b8a4ccb
running gpt-4o inference is now an action not a service
sriramk117 Dec 3, 2024
5363732
cleaned up some comments
sriramk117 Dec 3, 2024
d73c983
goal status cancellation
sriramk117 Dec 5, 2024
2326742
temporary changes for running testing procedures
sriramk117 Dec 11, 2024
b95ac8e
republisher.yaml reverted to original
sriramk117 Jan 2, 2025
4bf52ea
fixed cv2 visualization merge conflict
sriramk117 Jan 2, 2025
9382b67
segmentation inference optimization workin
sriramk117 Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
changes to segmentallitems node initializing it as a perception node
sriramk117 committed Nov 9, 2024

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
commit 23577ae9a3c494b2c991636ba00485135e0e34c6
1 change: 0 additions & 1 deletion ada_feeding_msgs/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -35,7 +35,6 @@ rosidl_generate_interfaces(${PROJECT_NAME}
"srv/AcquisitionReport.srv"
"srv/AcquisitionSelect.srv"
"srv/GetRobotState.srv"
"srv/ToggleFaceDetection.srv"
"srv/GenerateCaption.srv"
"srv/ModifyCollisionObject.srv"

Original file line number Diff line number Diff line change
@@ -171,6 +171,7 @@ def main(args=None):
from ada_feeding_perception.face_detection import FaceDetectionNode
from ada_feeding_perception.food_on_fork_detection import FoodOnForkDetectionNode
from ada_feeding_perception.segment_from_point import SegmentFromPointNode
from ada_feeding_perception.segment_all_items import SegmentAllItemsNode
from ada_feeding_perception.table_detection import TableDetectionNode

rclpy.init(args=args)
@@ -179,6 +180,7 @@ def main(args=None):
face_detection = FaceDetectionNode(node)
food_on_fork_detection = FoodOnForkDetectionNode(node)
segment_from_point = SegmentFromPointNode(node) # pylint: disable=unused-variable
segment_all_items = SegmentAllItemsNode(node) # pylint: disable=unused-variable
table_detection = TableDetectionNode(node)
executor = MultiThreadedExecutor(num_threads=16)

Loading