Connecting operator console MTMs to suture challenge AMBF environment?

[sunshineinsandiego]

Hi - I am trying to connect the MTMs in the surgical console (driven my a human operator) to the simulated PSMs in AMBF / the suture challenge environment. In this step, the directions say run dvrk-ros application for the dvrk_mtms to connect the output from the console MTMs to the simulated PSMs in the suture challenge simulation. But in the linked github repo, I don't see a dvrk_mtm code base.

How do I connect the two MTMs in the console to the two simulated PSMs in the suture challenge?

Thanks for your help!

[adnanmunawar]

Hello,

Have you taken a look at this document (https://github.com/surgical-robotics-ai/surgical_robotics_challenge/blob/master/docs/teleoperation_supplemental.md). There are linked instructions on how to run the MTM.

[sunshineinsandiego]

Hi! Thanks, definitely did not see that. Will take a look and report back with any questions. Thank you!

[sunshineinsandiego]

Following up on this as I am still not able to connect the MTMs to the suture challenge sim.

I currently have the operating console MTMs successfully connected to the physical PSMs of the DVRK. When I check the rostopic list, the operating console MTMs are broadcasting the required topics to move the physical PSMs of the DVRK.

However, I can not seem to figure out how to use those topics in the suture challenge simulation.

If I look at the document here: I have clearly passed Step 1. "Teleoperation Devices" because the operating console MTMs are working with the physical PSMs.

When I move to Step 2, I can successfully start the ambf simulator.

However, when I try to run the teleoperation scripts, I receive the following errors:

~/surgical_robotics_challenge/teleoperation$ ./mtm_psm_pair_teleop.sh 
Traceback (most recent call last):
  File "mtm_psm_pair_run.py", line 48, in <module>
    from surgical_robotics_challenge.simulation_manager import SimulationManager
  File "surgical_robotics_challenge/scripts/surgical_robotics_challenge/simulation_manager.py", line 1, in <module>
    from ambf_client import Client
ModuleNotFoundError: No module named 'ambf_client'

The above error occurs without sourcing the setup bash file. It is resolved but replaced by a different error when that bash file is sourced:

~/surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation$ source ~/ambf/build/devel/setup.bash
~/surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation$ ./mtm_psm_pair_teleop.sh 
Traceback (most recent call last):
  File "mtm_psm_pair_run.py", line 48, in <module>
    from surgical_robotics_challenge.simulation_manager import SimulationManager
  File "/surgical_robotics_challenge/scripts/surgical_robotics_challenge/simulation_manager.py", line 1, in <module>
    from ambf_client import Client
  File "/home/arclab/Documents/ambf/build/devel/lib/python3/dist-packages/ambf_client/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 45, in <module>
  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/__init__.py", line 49, in <module>
    from .client import spin, myargv, init_node, \
  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/client.py", line 52, in <module>
    import roslib
  File "/opt/ros/noetic/lib/python3/dist-packages/roslib/__init__.py", line 50, in <module>
    from roslib.launcher import load_manifest  # noqa: F401
  File "/opt/ros/noetic/lib/python3/dist-packages/roslib/launcher.py", line 42, in <module>
    import rospkg
ModuleNotFoundError: No module named 'rospkg'

So the ambf_client module not found error is resolved by sourcing the bash file from the ambf repo, but because the ros bash file isn't sourced, there's now a rospkg not found error. I can resolve the rospkg not found error by sourcing the ros bash file.

surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation$ source /opt/ros/noetic/setup.bash 
surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation$ ./mtm_psm_pair_teleop.sh 
Traceback (most recent call last):
  File "mtm_psm_pair_run.py", line 48, in <module>
    from surgical_robotics_challenge.simulation_manager import SimulationManager
  File "/surgical_robotics_challenge/scripts/surgical_robotics_challenge/simulation_manager.py", line 1, in <module>
    from ambf_client import Client
ModuleNotFoundError: No module named 'ambf_client'

But then the ambf_client module is not found.

How to resolve this?

Thanks!

[adnanmunawar]

Hello,

This is a problem of sourcing multiple ROS workspaces. It should be simple to fix, please check out the AMBF readme. I am copying the text here:

Working with Multiple ROS Workspaces: You may notice that if you have ROS packages in a Catkin workspace (e.g. ~/your_catkin_ws) then sourcing AMBF unsources that your_catkin_ws (i.e. the packages in your_catkin_ws are no longer accessible in the terminal) and vice-versa. This happens because the workspaces (your_catkin_ws and ambf/build/devel) are not properly overlaid. You can read more about workspace overlaying here: Overlaying with catkin workspaces.
In summary, if you wish to use AMBF's ROS packages (e.g. ambf_client or ambf_msgs) within a separate ROS package (e.g., your_catkin_ws), it is recommended to overlay the Catkin workspaces (other Catkin workspaces with AMBF):
Ensure that AMBF is sourced in your terminal.
Ensure your_catkin_ws has not been built (catkin clean and/or remove build and devel if it has been built).
Build your_catkin_ws.
Under the hood this will do a caktin workspace overlay, which will make sure AMBF is sourced whenever you source your_catkin_ws and that sourcing your_catkin_ws does not make the AMBF packages "unsource"/get overwritten in the environment variables.

[sunshineinsandiego]

OK, got it, so fixed the sourcing issues, but now have this issue:

$ /mtm_psm_pair_teleop.sh 
Specified Arguments
Namespace(client_name='mtm_sim_teleop', run_psm_one=True, run_psm_three=False, run_psm_two=True)
Found Common Object Namespace as:  /ambf/env/
LOADING CONTROLLER FOR  psm1
psm1_target_ik NAMED OBJECT NOT FOUND
psm1_palm_joint_ik NAMED OBJECT NOT FOUND
psm1_target_fk NAMED OBJECT NOT FOUND
LOADING CONTROLLER FOR  psm2
psm2_target_ik NAMED OBJECT NOT FOUND
psm2_palm_joint_ik NAMED OBJECT NOT FOUND
psm2_target_fk NAMED OBJECT NOT FOUND
(None, None, None)
(None, None, None)
Exception! Message Type: %s CRTK CP MESSAGE TYPE IS NEITHER PoseStamped or TransformStamped None
Traceback (most recent call last):
  File "mtm_psm_pair_run.py", line 254, in <module>
    leader_l = MTM('/MTML/')
  File "/home/arclab/Documents/surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation/input_devices/mtm_device_crtk.py", line 202, in __init__
    self.MEASURED_CP_MESSAGE_TYPE = get_crtk_cp_msg_type_from_str(rostopic.get_topic_type(pose_sub_topic_name)[0])
  File "/home/arclab/Documents/surgical_robotics_challenge/scripts/surgical_robotics_challenge/teleoperation/input_devices/mtm_device_crtk.py", line 156, in get_crtk_cp_msg_type_from_str
    raise TypeError
TypeError
Closing publisher for:  /ambf/env/Plane
Closing publisher for:  /ambf/env/lights/light2
Closing publisher for:  /ambf/env/cameras/cameraL
Closing publisher for:  /ambf/env/cameras/cameraR
Closing publisher for:  /ambf/env/cameras/cameraL2
Closing publisher for:  World
Closing publisher for:  /ambf/env/psm1/baselink
Closing publisher for:  /ambf/env/psm1/toolyawlink
Closing publisher for:  /ambf/env/psm1/Sensor0
Closing publisher for:  /ambf/env/psm1/Actuator0
Closing publisher for:  /ambf/env/psm2/baselink
Closing publisher for:  /ambf/env/psm2/toolyawlink
Closing publisher for:  /ambf/env/psm2/Sensor0
Closing publisher for:  /ambf/env/psm2/Actuator0
Closing publisher for:  /ambf/env/BlackGround
Closing publisher for:  /ambf/env/CameraFrame
Closing publisher for:  /ambf/env/Ledge
Closing publisher for:  /ambf/env/Needle
Closing publisher for:  /ambf/env/Phantom
Closing publisher for:  /ambf/env/Barrier
Closing publisher for:  /ambf/env/Entry1
Closing publisher for:  /ambf/env/Entry2
Closing publisher for:  /ambf/env/Entry3
Closing publisher for:  /ambf/env/Entry4
Closing publisher for:  /ambf/env/Exit1
Closing publisher for:  /ambf/env/Exit2
Closing publisher for:  /ambf/env/Exit3
Closing publisher for:  /ambf/env/Exit4

This brings up the following question:

• Which ros topics does the simulator subscribe to that are published by the operating console?

Thanks!