Real Robot Labs

Working with the TurtleBot3 Waffles

Introduction

You can use these face-to-face lab sessions to have a go at implementing some of the things that you have learnt in simulation so far.

Social Distancing

Under the current circumstances, it is important that you maintain social distancing throughout these sessions:

• Remain at the workstation that you have been assigned.
• Only work with the laptop that you have been provided with.
• Maintain a 2 meter distance with anyone else at all times.
• Raise your hand if you have any issues.
• The robots will remain in the robot arena at the centre of the computer room and the teaching staff will handle them for you, if required.

Note: Any breach of these rules will be considered a health and safety violation, which will be reported and will result in you being asked to leave the session.

Working with the Laptops (Please Read this Carefully)

The first important thing to note is that so far you have been working with ROS Melodic and Ubuntu 18.04 in simulation. The real robots and laptops that you will be working with in these labs run on a slightly older version of both Ubuntu and ROS: ROS Kinetic and Ubuntu 16.04. As a result, there will be slight differences between some of the commands that you will use when working with ROS here, compared with what you have done in simulation.

OS/Software Updates

Whilst working on these laptops you may at some point see a message asking if you would like to install software updates (as below) or update the operating system to the latest version. Please do not perform any sort of updates on this laptop, and just dismiss any messages that may appear:

WiFi

Robots and Laptops communicate with one another on a special wireless network running in the Diamond called DIA-LAB. Laptops must be connected to the DIA-LAB network in order for you to communicate with the robots, so make sure that this is the case by clicking on the WiFi symbol in the top right-hand corner of the laptop screen. This is an internal network, so you will not be able to access the internet and you should use the Computer Room 3 workstations to access any external resources (such as this Wiki) instead.

Credentials for DIA-LAB should have already been set on the laptops, allowing you to connect straight away, but speak to a GTA if you are having any issues.

Robot/Laptop Pairings

The laptop you are working with is already paired with a specific TurtleBot3 Waffle robot in the robot arena. Identify your Laptop Number by opening up an Ubuntu terminal instance on the laptop using the Ctrl+Alt+T keyboard shortcut. The command prompt should look similar to this:

robot@dia-laptopX:~$

The X indicates your Laptop Number. Then have a look at this document to see which Waffle Robot your laptop is paired with (you will need to use your University of Sheffield login details to access the link).

Check that the Laptop is in Waffle Mode

Note: The steps below apply to some - but not all - of the laptops, so please read the following carefully to check whether this applies to you...

Depending on the laptop you are using, you may be presented with text similar to the following when you open a terminal window on the laptop. Note the second line (as highlighted) Current robot is set to ...:

If the current robot is set to MiRo, then you will need to run the following command in order to switch back to Waffle Mode:

$ robot_switch turtlebot3

You should then see the following text, note again the Current robot is set to ... line and ensure that this now reads Turtlebot3:

Then, close the terminal window (don’t worry if you see a warning message saying that there is a process still running, just click "Close Terminal" to proceed). Every new terminal window you open from now on should have the correct settings applied, so you can now carry on working.

Available IDE

Spyder is installed on the robot laptops if you want to use this to write/edit Python code. This is part of the Anaconda-Python installation so in order to launch it, follow the steps below:

1. Open a new terminal and activate conda:

    $ conda activate
   

   (base) should appear at the start of the terminal prompt.

2. To then launch the Spyder IDE enter:

    $ spyder
   

Working with the Robots and ROS

Exercise 1: Getting started

There are a number of initial steps that need to be carried out in order to start up your robot and get ROS running.

Note: As you will have experienced whilst working with ROS in simulation over the last six weeks: you will often need to have several terminal windows running simultaneously. Only close down terminals or shutdown processes (using Ctrl+C) when instructed to do so otherwise your robot may not function correctly!

We will be accessing the robot over the DIA-LAB WiFi network via a secure shell connection from the laptop, allowing us to access the the Linux terminal on the robot computer. We do this using ssh.

Remember to make sure the laptop is connected to the DIA-LAB WiFi network!

1. Open a new terminal window on the laptop by using the Ctrl+Alt+T keyboard shortcut (or otherwise). We will refer to this as [TERMINAL 1].

2. Enter the following to establish a connection to your robot via ssh:

    [TERMINAL 1]: $ ssh waffle@dia-waffleX
   

   Replace X with the number of the robot that your laptop is paired with. Enter the password when requested.

3. The terminal prompt should change as follows:

    waffle@dia-waffleX:~$  
   

   This indicates that you are now logged in to the computer on the robot (dia-waffleX) via the local username: waffle

4. Then launch a tmux session to provide some advanced functionality for the remote session:

    [TERMINAL 1]: $ tmux
   

   Any text in the terminal should now clear and a green banner should appear across the bottom of the terminal window indicating that you are now working within a tmux session. tmux gives us a more robust connection to the robot and you can find a (very quick) guide for how to work with it here. We will refer to this terminal window as [TERMINAL 1][TMUX 0] for the remainder of these instructions.

5. To get ROS up and running we need to launch the ROS master on the laptop. Open up a new terminal window on the laptop using Ctrl+Alt+T (herein referred to as [TERMINAL 2]), and then launch roscore:

    [TERMINAL 2]: $ roscore
   

   A terminal message ending as follows:

    started core services [/rosout]  
   

   indicates that the ROS Master has been launched successfully.

6. We then need to launch all the core packages on the robot and we can do this using a single launch file located within a convenience package called turtlebot3_bringup, which is installed locally on the robot. Return to your tmux session in [TERMINAL 1] and enter the following:

    [TERMINAL 1][TMUX 0]: $ roslaunch turtlebot3_bringup turtlebot3_robot.launch
   

   Once you see a message containing the text:

    Calibration End  
   

   then the robot core packages are up and running.

7. To help manage all the terminal windows that are running ROS processes on the laptop, it's good to keep all of them together by using tabs within [TERMINAL 2]. Go to [TERMINAL 2] where roscore is running (and which will now be referred to as [TERMINAL 2][TAB 1]) and create a new tab using the Ctrl+Shift+T keyboard shortcut (we will refer to this new tab as [TERMINAL 2][TAB 2] from here on in).

8. Finally, we need to launch some core packages on the laptop too. This is done in the same way as on the robot using the turtlebot3_bringup package again (which is installed locally on the laptop as well). This time however, we will point the roslaunch command to a slightly different launch file. In [TERMINAL 2][TAB 2] enter the following:

    [TERMINAL 2][TAB 2]: $ roslaunch turtlebot3_bringup turtlebot3_remote.launch
   

   The following message should appear to indicate that the remote packages on the laptop have also been launched successfully:

    process[robot_state_publisher-1]: started with pid [xxxx]  
   

The connection between our robot and the laptop is now established, the ROS master is up and running on the laptop and a number of ROS Nodes have been launched on both the robot and laptop, so we can now start working with the robot!

Exercise 2: Observing your Robot's Environment

You may not be able to see the robot arena, or your robot, from where you are sitting in the computer room, so it's important to use some of the ROS tools that you will now be familiar with in order to keep an eye on the environment as you move the robot around. Remember that there will be lots of other robots in the arena, so take care at all times!

1. First, open a new tmux tab on the robot using the Ctrl+B, C key combination ([TERMINAL 1][TMUX 1]).

2. Then, launch some of the camera nodes on the robot (and start publishing image data to ROS topics):

    [TERMINAL 1][TMUX 1] $ roslaunch turtlebot3_bringup turtlebot3_realsense.launch
   

   The message:

    [ INFO] [###.###]: RealSense Node Is Up!
   

   indicates that the camera nodes are up and running and image data is being streamed to ROS topics on the network.

3. Remember from Week 3 that RViz is a ROS tool that allows us to visualise the data being measured by a robot in real-time. This can be launched on the laptop as follows... Open a new terminal instance on the laptop ([TERMINAL 3]) and enter the following to launch Rviz and a model of the robot:

    [TERMINAL 3]: $ rosrun rviz rviz -d `rospack find turtlebot3_description`/rviz/model.rviz
   

   

4. Like we did in Week 6, we can also use the rqt_image_view node to view live image data being streamed to ROS image topics. You could open another terminal instance on the laptop to launch this ([TERMINAL 4]):

    [TERMINAL 4] $ rosrun rqt_image_view rqt_image_view
   

   Select /camera/color/image_raw in the dropdown topic list to see the images being obtained and published by the robot's camera and associated camera nodes.

Use these tools to keep an eye on your robot's environment whilst performing the next exercise.

Exercise 3: Driving your robot around using the keyboard

In simulation you used the turtlebot3_teleop package to drive your robot around in an empty world. This works in exactly the same way with a real robot in a real world!

1. Open a new tab in [TERMINAL 2] (which we will call [TERMINAL 2][TAB 3]) and enter exactly the same roslaunch command as you used in simulation to launch the turtlebot3_teleop node:

    [TERMINAL 2][TAB 3]: $ roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch
   

2. Drive your robot around using the laptop keyboard (as you did in simulation) taking care to avoid any obstacles as you do!

3. Once you've spent a bit of time on this, close the process down by entering Ctrl+C in [TERMINAL 2][TAB 3] but leave the tab open.

Exercise 4: Using SLAM to Create a Map of the Environment

Remember how we used SLAM in Week 3 to create a map of a simulated environment? We'll do this now on a real robot in a real environment (i.e. the arena in the computer room that you are currently sat in!)

1. Make sure that the ROS master is still up and running on the laptop in [TERMINAL 2][TAB 1], and the robot packages are still running in the two tmux instances in [TERMINAL 1].

2. Return to [TERMINAL 2][TAB 2] and enter Ctrl+C to stop the remote bringup processes that should still be running there (if they aren't then that's fine, just leave it as it is).

3. Close down RViz and the rqt_image_view nodes running in [TERMINAL 3] and [TERMINAL 4] as well, you shouldn't need these for this.

4. Return to [TERMINAL 2][TAB 2] and enter the following command to launch all the necessary SLAM nodes on the laptop:

    [TERMINAL 2][TAB 2]: $ roslaunch turtlebot3_slam turtlebot3_slam.launch slam_methods:=gmapping
   

   This will launch RViz again, where you should now be able to see a model of the TurtleBot3 from a top-down view surrounded by green dots representing the real-time LiDAR data. The SLAM tools will already have begun processing this data to start building a map of the boundaries that are currently visible to your robot based on its position in the environment.

   

   Note: To begin with your robot may just appear as a white shadow (similar to the left-hand image). It may take some time for the robot to render correctly (like the right-hand image) as the SLAM processes and data communications catch up with one another. This can sometimes take up to a minute or so, so please be patient! If - after 1 minute - nothing has happened, then speak to a GTA.

5. Return to [TERMINAL 2][TAB 3], launch the turtlebot3_teleop node again and start to drive the robot around slowly and carefully to build up a complete map of the area.

    [TERMINAL 2][TAB 3]: $ roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch
   

   Note: It's best to do this slowly and perform multiple circuits of the whole area to build up a more accurate map.

6. Once you are happy that your robot has built up a good map of its environment, you can save this map using the map_server package (again, in exactly the same way as you did in Week 3):

   1. First, create a new directory on the laptop to store your data in. Stop the teleop node in [TERMINAL 2][TAB 3] and stay in the same terminal instance to do this:

       [TERMINAL 2][TAB 3]: $ mkdir -p ~/com2009/2021/[your sheffield username]
      

   2. Then, use rosrun to run the map_saver node from the map_server package to save a copy of your map:

       [TERMINAL 2][TAB 3]: $ rosrun map_server map_saver -f ~/com2009/2021/[your sheffield username]/[map name]
      

      Replacing [map name] with an appropriate name for your map. This will create two files: a [map name].pgm and a [map name].yaml file, both of which contain data related to the map that you have just created.

   3. The .pgm file can be opened in eog on the laptop. Either navigate to the file in the Ubuntu File Explorer and double click it, or navigate to the file in the terminal and open it using eog that way. to view an image of the map you have just created.

7. Return to [TERMINAL 2][TAB 2] and close down SLAM by pressing Ctrl+C. The process should stop and RViz should close down.

Exercise 5: Writing your own ROS Nodes

Have a go at writing a ROS node (in Python) to control your robot now. Why not try recreating the move_circle node that you developed in Week 2?

First, you will need to create a new ROS package on the laptop using catkin_create_pkg:

As you should know by now, you need to build ROS packages in the Catkin Workspace and develop your nodes within these packages. A Catkin Workspace exists on both the robot and laptop. On the laptop it's here:

/home/robot/catkin_ws/src/

And on the robot it's here:

/home/waffle/catkin_ws/src/

As far as ROS is concerned, it doesn't matter whether you run your nodes on the robot or the laptop, as all the messages on our ROS system are standardised and published to topics that are visible system-wide (as managed by the ROS Master). As a result, any nodes running on your robot or laptop can subscribe, or publish, to any topic message that they need to. When dealing with real-time control, it may be more effective to run nodes locally (i.e. on the robot) rather than remotely (from the laptop), but for simplicity, we recommend creating your package and developing your nodes on the laptop first and then you can transfer the package over to the robot later following the steps for moving files and folders between filesystems, if you want to.

1. In [TERMINAL 2][TAB 2] navigate to the catkin_ws/src/ directory on the laptop:

    [TERMINAL 2][TAB 2]: $ cd ~/catkin_ws/src
   

2. Use the catkin_create_pkg tool (as you have done in simulation) to create a new ROS package:

    [TERMINAL 2][TAB 2]: $ catkin_create_pkg [your sheffield username] rospy
   

3. Navigate to the src directory of the package that you have just created. Create a new Python file, make it executable, and start programming!

When you are ready, you can launch your node using the rosrun command (remember: rosrun [package name] [script name]), or you could even create a launch file to launch your node using roslaunch too.

Lab Exercises Complete. Please follow the steps below to shutdown your robot correctly before leaving.

Finishing Up

1. First, close down all the active processes that are still running on both the laptop and the robot by checking through all of your open terminal windows/tabs (including all the active windows in tmux) and closing them down using Ctrl+C.

2. Disconnect from the tmux session running in [TERMINAL 1] by going to any of your tmux windows in [TERMINAL 1] and entering the Ctrl+B,D keyboard combination to disconnect and return to the ssh session.

3. Then, shutdown your robot by entering the following command in the ssh session in [TERMINAL 1]:

    [TERMINAL 1]: $ sudo shutdown -P now
   

   Enter the password when asked then wait for the "Connection to dia-waffleX closed" message.

4. Ask a GTA to then turn off your robot for you.

5. Finally, shutdown the laptop and leave it where it is on the workstation that you have been working at today.

How-Tos

Using tmux

The keyboard commands in tmux must be prefixed with Ctrl+B, followed by the command key for the action that you want to perform. Press Ctrl+B simultaneously, release and then press the command key. Some useful commands are as follows, with a full list available on the tmux manual pages:

• Ctrl+B,C: Create a new window
• Ctrl+B,N: Move to the next window (right)
• Ctrl+B,P: Move to the previous window (left)
• Ctrl+B,[0 to 9]: Select window by index 0 to 9
• Ctrl+B,&: Close the current window
• Ctrl+B,D: Disconnect from the tmux session

Moving files between the Laptop and Robot

The laptop and robot will communicate with one another on their own ROS network. In theory then, you can run nodes directly on the laptop in order to control the robot, but you can also run nodes and create packages on the robot filesystem too. If you want to transfer a file/script and run it locally (on the robot), or copy some data back from the robot to the laptop, then use scp as follows:

Copy a file from the Laptop to the Robot

1. Open a terminal window on the laptop and navigate to the location of the file that you want to transfer:

    $ cd path/to/file
   

2. To transfer this file to the robot filesystem use scp as follows:

    $ scp [file] waffle@dia-waffleX:/home/waffle/[location]/[file]
   

   Where [file] is the name of the file that you want to copy, dia-waffleX is the name of the robot that your laptop is paired to (as above) and [location] is the path to the directory you want to copy the file to on the robot's filesystem (note that this is relative to the home directory of the robot: /home/waffle/...).

3. Then, enter the password for the robot when asked and the transfer process will start.

Copy a file from the Robot to the Laptop

The same works the other way, if you want to copy a file back from the robot to the laptop (via a terminal instance running on the laptop):

$ scp waffle@dia-waffleX:/home/waffle/[location]/[file] /home/robot/[file] 

You may be asked to enter the password for the robot again.

(Confusingly, the laptop username is robot hence the home directory on the laptop is /home/robot/)

Copy a folder (from the Laptop to the Robot)

To copy a folder (directory) from the laptop to the robot, use the -r option with scp:

$ scp -r path/to/directory waffle@dia-waffleX:/home/waffle/[location]/