Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSDK-4051 tp space should be able to continuously sample ptgs #2792

Conversation

biotinker
Copy link
Member

@biotinker biotinker commented Aug 16, 2023

This is a huge PR that fundamentally reworks the way that PTGs are used.

All computable PTGs have been given a Transform function allowing a given pose on a trajectory to be computed. A ptg IK frame struct has been created allowing our IK solvers to solve a given PTG for a pose. The precomputed grid simulation of PTGs has been removed, instead we now precompute only the endpoint of each PTG trajectory.

TP-space RRT now solves bidirectionally, and is highly effective at solving for orientation. This also solves RSDK-3598. Path smoothing has been implemented for TP-space paths and is able to improve the paths of TP-space as part of a normal motion planning call.

Most testing of this as of this writing has centered around the TP-space algorithm itself. I have not tested this on hardware yet. I would like to write a few more *_test.go files as well.

More documentation of code, and passes removing old obsolete comments, will be done in the next few commits.

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Aug 16, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 16, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 16, 2023
@biotinker biotinker marked this pull request as ready for review August 16, 2023 21:16
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
}

func (pf *ptgIKFrame) Geometries(inputs []referenceframe.Input) (*referenceframe.GeometriesInFrame, error) {
return nil, errors.New("geometries not implemented for ptg IK frame")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't they be?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because this frame is only used for IK. We care about Geometries in the PTG group frame, where they are implemented.

Probably the constructor here should be private to make sure no one uses these elsewhere inadvertently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused why we even need this structure to be honest

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like why can't we just use the group frame to do this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 3dof structure of the group frame does not play nicely with the IK solver, on account of the IK solver assuming a continuous distribution of float change -> position gradient, while the first Input to a group frame is an integer representing which PTG family to use. We want the 3dof frame for actually interacting with the base and forming the plan, but a 2dof frame for solving.

@biotinker biotinker requested a review from raybjork August 23, 2023 22:38
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 23, 2023
@github-actions
Copy link
Contributor

Availability

Scene # viamrobotics:main biotinker:20230717_RSDK-4051-tp-space-should-be-able-to-continuously-sample-pt-gs Percent Improvement Health
1 100% 100% 0%
2 100% 100% 0%
3 100% 100% 0%
4 100% 100% 0%
5 20% 30% 50%
6 30% 10% -67%
7 60% 50% -17%
8 100% 100% 0%
9 0% 0% NaN%
10 100% 100% 0%
11 100% 100% 0%
12 100% 100% 0%
13 100% 100% 0%
14 100% 100% 0%
15 100% 100% 0%
16 0% 0% NaN%
17 60% 50% -17%
18 0% 0% NaN%

Quality

Scene # viamrobotics:main biotinker:20230717_RSDK-4051-tp-space-should-be-able-to-continuously-sample-pt-gs Percent Improvement Probability of Improvement Health
1 1.31±0.00 1.31±0.00 -0% 50%
2 0.90±0.00 0.90±0.00 -0% 50%
3 2.44±0.01 2.44±0.01 -0% 50%
4 3.56±0.76 4.03±0.46 -13% 30%
5 8.52±1.62 8.81±1.38 -3% 45%
6 10.50±2.34 7.90±0.00 25% 87%
7 3.03±0.96 3.42±0.75 -13% 37%
8 4.48±0.86 4.55±0.84 -1% 48%
9 NaN±NaN NaN±NaN NaN% NaN%
10 4.45±0.49 4.52±0.55 -2% 46%
11 3.13±0.00 3.13±0.00 -0% 43%
12 3.91±0.85 3.91±0.85 -0% 50%
13 654.51±0.00 654.51±0.00 -0% 45%
14 1601.01±43.83 1601.01±43.83 -0% 50%
15 22435.67±0.00 22435.67±0.00 0% 55%
16 NaN±NaN NaN±NaN NaN% NaN%
17 10868.96±1477.19 11064.90±1469.85 -2% 46%
18 NaN±NaN NaN±NaN NaN% NaN%

Performance

Scene # viamrobotics:main biotinker:20230717_RSDK-4051-tp-space-should-be-able-to-continuously-sample-pt-gs Percent Improvement Probability of Improvement Health
1 5.50±2.87 5.50±2.87 -0% 50%
2 5.50±2.87 5.50±2.87 -0% 50%
3 5.50±2.87 5.50±2.87 -0% 50%
4 5.50±2.87 5.50±2.87 -0% 50%
5 4.50±3.50 4.33±2.87 4% 51%
6 7.00±0.82 7.00±0.00 -0% 50%
7 4.17±2.11 5.00±2.83 -20% 41%
8 5.50±2.87 5.50±2.87 -0% 50%
9 NaN±NaN NaN±NaN NaN% NaN%
10 5.50±2.87 5.50±2.87 -0% 50%
11 5.50±2.87 5.50±2.87 -0% 50%
12 5.50±2.87 5.50±2.87 -0% 50%
13 5.50±2.87 5.50±2.87 -0% 50%
14 5.50±2.87 5.50±2.87 -0% 50%
15 5.50±2.87 5.50±2.87 -0% 50%
16 NaN±NaN NaN±NaN NaN% NaN%
17 4.67±2.87 5.00±3.03 -7% 47%
18 NaN±NaN NaN±NaN NaN% NaN%

The above data was generated by running scenes defined in the motion-testing repository
The SHA1 for viamrobotics:main is: 19c2e624a346b922592b40b57d2a68b918d70e92
The SHA1 for biotinker:20230717_RSDK-4051-tp-space-should-be-able-to-continuously-sample-pt-gs is: 19c2e624a346b922592b40b57d2a68b918d70e92

  • 10 samples were taken for each scene
  • A timeout of 5.0 seconds was imposed for each trial

@github-actions
Copy link
Contributor

Code Coverage

Code Coverage
Package Line Rate Delta Health
go.viam.com/rdk/cli 4% 0.00%
go.viam.com/rdk/components/arm 59% 0.00%
go.viam.com/rdk/components/arm/fake 27% 0.00%
go.viam.com/rdk/components/arm/universalrobots 42% 0.00%
go.viam.com/rdk/components/arm/wrapper 19% 0.00%
go.viam.com/rdk/components/arm/xarm 22% 0.00%
go.viam.com/rdk/components/arm/yahboom 6% 0.00%
go.viam.com/rdk/components/audioinput 43% -0.47%
go.viam.com/rdk/components/base 60% 0.00%
go.viam.com/rdk/components/base/agilex 60% 0.00%
go.viam.com/rdk/components/base/kinematicbase 43% +0.20%
go.viam.com/rdk/components/base/sensorbase 68% 0.00%
go.viam.com/rdk/components/base/wheeled 89% 0.00%
go.viam.com/rdk/components/board 60% 0.00%
go.viam.com/rdk/components/board/customlinux 50% 0.00%
go.viam.com/rdk/components/board/fake 38% 0.00%
go.viam.com/rdk/components/board/genericlinux 8% 0.00%
go.viam.com/rdk/components/board/numato 19% 0.00%
go.viam.com/rdk/components/board/pi/impl 11% 0.00%
go.viam.com/rdk/components/camera 57% 0.00%
go.viam.com/rdk/components/camera/align 58% 0.00%
go.viam.com/rdk/components/camera/fake 74% 0.00%
go.viam.com/rdk/components/camera/ffmpeg 82% 0.00%
go.viam.com/rdk/components/camera/replaypcd 90% 0.00%
go.viam.com/rdk/components/camera/rtsp 52% 0.00%
go.viam.com/rdk/components/camera/transformpipeline 73% 0.00%
go.viam.com/rdk/components/camera/ultrasonic 61% 0.00%
go.viam.com/rdk/components/camera/videosource 39% 0.00%
go.viam.com/rdk/components/encoder 57% 0.00%
go.viam.com/rdk/components/encoder/ams 63% 0.00%
go.viam.com/rdk/components/encoder/fake 83% 0.00%
go.viam.com/rdk/components/encoder/incremental 80% 0.00%
go.viam.com/rdk/components/encoder/single 86% 0.00%
go.viam.com/rdk/components/gantry 60% 0.00%
go.viam.com/rdk/components/gantry/multiaxis 82% 0.00%
go.viam.com/rdk/components/gantry/singleaxis 81% 0.00%
go.viam.com/rdk/components/generic 79% 0.00%
go.viam.com/rdk/components/gripper 56% 0.00%
go.viam.com/rdk/components/input 88% 0.00%
go.viam.com/rdk/components/input/fake 94% +1.49%
go.viam.com/rdk/components/input/gpio 85% 0.00%
go.viam.com/rdk/components/motor 71% 0.00%
go.viam.com/rdk/components/motor/dimensionengineering 67% 0.00%
go.viam.com/rdk/components/motor/dmc4000 70% 0.00%
go.viam.com/rdk/components/motor/fake 55% 0.00%
go.viam.com/rdk/components/motor/gpio 73% 0.00%
go.viam.com/rdk/components/motor/gpiostepper 66% 0.00%
go.viam.com/rdk/components/motor/tmcstepper 53% 0.00%
go.viam.com/rdk/components/motor/ulnstepper 53% 0.00%
go.viam.com/rdk/components/movementsensor 74% 0.00%
go.viam.com/rdk/components/movementsensor/adxl345 75% 0.00%
go.viam.com/rdk/components/movementsensor/gpsnmea 56% 0.00%
go.viam.com/rdk/components/movementsensor/gpsrtkpmtk 33% 0.00%
go.viam.com/rdk/components/movementsensor/gpsrtkserial 29% 0.00%
go.viam.com/rdk/components/movementsensor/merged 91% 0.00%
go.viam.com/rdk/components/movementsensor/mpu6050 84% 0.00%
go.viam.com/rdk/components/movementsensor/replay 93% 0.00%
go.viam.com/rdk/components/movementsensor/rtkutils 24% 0.00%
go.viam.com/rdk/components/movementsensor/wheeledodometry 76% 0.00%
go.viam.com/rdk/components/posetracker 71% 0.00%
go.viam.com/rdk/components/powersensor 49% 0.00%
go.viam.com/rdk/components/sensor 51% 0.00%
go.viam.com/rdk/components/sensor/ultrasonic 38% 0.00%
go.viam.com/rdk/components/servo 62% 0.00%
go.viam.com/rdk/components/servo/gpio 72% 0.00%
go.viam.com/rdk/config 80% 0.00%
go.viam.com/rdk/control 57% 0.00%
go.viam.com/rdk/data 73% 0.00%
go.viam.com/rdk/examples/customresources/demos/remoteserver 0% 0.00%
go.viam.com/rdk/grpc 25% 0.00%
go.viam.com/rdk/internal/cloud 100% 0.00%
go.viam.com/rdk/ml 67% 0.00%
go.viam.com/rdk/ml/inference 67% 0.00%
go.viam.com/rdk/module 75% 0.00%
go.viam.com/rdk/module/modmanager 82% 0.00%
go.viam.com/rdk/motionplan 78% -1.34%
go.viam.com/rdk/motionplan/ik 68% N/A
go.viam.com/rdk/motionplan/tpspace 52% -25.98%
go.viam.com/rdk/operation 83% 0.00%
go.viam.com/rdk/pointcloud 67% 0.00%
go.viam.com/rdk/protoutils 49% 0.00%
go.viam.com/rdk/referenceframe 69% +3.59%
go.viam.com/rdk/resource 77% 0.00%
go.viam.com/rdk/rimage 55% 0.00%
go.viam.com/rdk/rimage/depthadapter 94% 0.00%
go.viam.com/rdk/rimage/transform 72% +0.09%
go.viam.com/rdk/rimage/transform/cmd/extrinsic_calibration 67% 0.00%
go.viam.com/rdk/robot 85% 0.00%
go.viam.com/rdk/robot/client 82% 0.00%
go.viam.com/rdk/robot/framesystem 36% 0.00%
go.viam.com/rdk/robot/impl 83% 0.00%
go.viam.com/rdk/robot/packages 75% 0.00%
go.viam.com/rdk/robot/server 55% 0.00%
go.viam.com/rdk/robot/web 65% +0.16%
go.viam.com/rdk/robot/web/stream 87% 0.00%
go.viam.com/rdk/services/baseremotecontrol 50% 0.00%
go.viam.com/rdk/services/baseremotecontrol/builtin 81% 0.00%
go.viam.com/rdk/services/datamanager 65% 0.00%
go.viam.com/rdk/services/datamanager/builtin 89% 0.00%
go.viam.com/rdk/services/datamanager/datacapture 73% 0.00%
go.viam.com/rdk/services/datamanager/datasync 0% 0.00%
go.viam.com/rdk/services/mlmodel 76% 0.00%
go.viam.com/rdk/services/mlmodel/tflitecpu 85% 0.00%
go.viam.com/rdk/services/motion 52% 0.00%
go.viam.com/rdk/services/motion/builtin 81% 0.00%
go.viam.com/rdk/services/navigation 47% 0.00%
go.viam.com/rdk/services/navigation/builtin 86% 0.00%
go.viam.com/rdk/services/sensors 81% 0.00%
go.viam.com/rdk/services/sensors/builtin 97% 0.00%
go.viam.com/rdk/services/shell 11% 0.00%
go.viam.com/rdk/services/slam 73% 0.00%
go.viam.com/rdk/services/slam/fake 82% 0.00%
go.viam.com/rdk/services/vision 34% 0.00%
go.viam.com/rdk/services/vision/colordetector 56% 0.00%
go.viam.com/rdk/services/vision/detectionstosegments 67% 0.00%
go.viam.com/rdk/services/vision/mlvision 64% -0.27%
go.viam.com/rdk/services/vision/obstaclesdepth 74% 0.00%
go.viam.com/rdk/services/vision/obstaclesdistance 77% 0.00%
go.viam.com/rdk/services/vision/obstaclespointcloud 59% 0.00%
go.viam.com/rdk/session 94% 0.00%
go.viam.com/rdk/spatialmath 82% -0.09%
go.viam.com/rdk/utils 65% 0.00%
go.viam.com/rdk/utils/contextutils 38% 0.00%
go.viam.com/rdk/vision 37% 0.00%
go.viam.com/rdk/vision/delaunay 87% 0.00%
go.viam.com/rdk/vision/keypoints 92% 0.00%
go.viam.com/rdk/vision/objectdetection 69% 0.00%
go.viam.com/rdk/vision/segmentation 54% 0.00%
go.viam.com/rdk/web/server 26% 0.00%
Summary 62% (23192 / 37594) -0.82%

defer close(m1chan)
defer close(m2chan)

dist := math.Sqrt(mp.planOpts.DistanceFunc(&ik.Segment{StartPosition: startPose, EndPosition: goalPose}))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the distance function either be responsible for taking the sqrt if necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally yes. In this case we're just defining a region within which to sample, and it's better to under-estimate than over estimate. This is the only place we sqrt the distance, and it's deliberate here.

nm.parallelNeighbors = 10

var successNode node
// Get the distance function that will find the nearest RRT map node in TP-space of *this* PTG
ptgDistOpt := mp.algOpts.distOptions[curPtg]
if invert {
ptgDistOpt = mp.algOpts.invertDistOptions[curPtg]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why the invert bool exists but it seems really hacky and I wonder if theres a better way to do this. Can't we just get the invert distance by sampleNode->treeNode and get normal distance by going treeNode->sample? So instead of adding a bool to these functions just change the order of the arguments being passed in?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this is really hacky. I'll brainstorm different ways to do this.

Can't we just get the invert distance by sampleNode->treeNode and get normal distance by going treeNode->sample?

That's what I originally implemented; unfortunately it doesn't work, as that's not quite how the problem needs to be formulated, due to the directionality of PTGs. You can't just invert the pose; you have to say "I need a pose X, such that when I compose pose A with the inverse of pose X, yields something close to pose B".

There may be a way to get it to fit together differently but my first three attempts failed.

}
lastDist = trajPt.Dist
}
if mp.algOpts.pathdebug {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to be committing this into main?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been SO darn useful and is the fourth time I've had to implement and then remove it. I think it's worth hiding behind an internal feature flag.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the idea actually and think if we're going to commit to it here we should also slowly work this in elsewhere

Copy link
Member

@raybjork raybjork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot going on here and I'm good with the general shape of it. Its going to be a lot easier to iterate on it once its merged in rather than trying to change it too much in review, so I'm good to merge with the understanding that we'll be doing some future passes on it soon

@biotinker biotinker merged commit 32c9ebe into viamrobotics:main Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants