Roadmap

Currently working on bold

---

Current Focus

Visual Cone Detection

• setup AirSim
• add cones position information
• dataset generation pipeline
• augmentation
  • noise
  • low light
  • weather
• generate dataset
• train YOLOv3-tiny
  • see here
• test inference
• test inference performance on Xavier
• annotate real data
• retrain the network
• test in real-world
• add sub-pixel refinement

Visual SLAM Exploration

• test orbslam 2
  • discarded as not robust enough
• test orbslam_dwo
  • discarded since no RT branch available, also slower than original orbslam
• test rovio
  • works OK on servo data
• test servo on provided data
  • bugfixes - doesn't compile
  • compile
  • test ROS node on provided data - works fine

Visual SLAM System

• continue working with servo
• integrate with the main system
• calibrate our cameras and IMU
• test on our data
• performance optimization
  • integrate with orbslam2-gpu
  • add binary vocabulary
  • remove Pangolin GUI
• publish points and poses to ROS
• test cone detection
• implement cone detector and descriptor
• implement data fusion in orbslam
• test on the car

---

Additional Directions

System

• define ROS node topology
• add documentation

Simulator

• eufs_sim - gazebo simulator
  • compiled
  • bugfix - no car
• AirSim - Microsoft's framework
  • test on linux
  • add lidar, see here
  • add IMU, see here
  • integrate with ROS, see here

Hardware

• camera
• LIDAR
• Nvidia Jetson setup
• capture with rosbag
• install and run on the car
• capture demo content
• capture real content
• Nvidia Drive PX2 setup
• after meeting with Nvidia decided to use Jetson AGX Xavier
• Nvidia Jetson Xavier setup
• install and run on the car
• TODO: update regarding the electric car
  • sensors
  • control

LIDAR Setup

• install ROS velodyne drivers
• test and store data
• control LIDAR speed from ROS, see here
• filter LIDAR by FOV and distance, search for ring information, see here
• LIDAR-camera calibration
  • lidar_camera_calibration doesn't compile, todo: debug
  • try this
• fusion with camera

LIDAR Cone Detection

• test the data
• add node to ROS
• ground removal
• cone segmentation and clustering
• cone tracking
  • probably redundant
• additional filtering
• color from intensity
• color from camera

LIDAR SLAM

• test off the shelf LIDAR SLAM solutions
  • loam_velodyne - not good enough for tracking cones
  • hdl_graph_slam - doesn't compile
  • cartographer
  • mrpt
  • gmapping
  • hector_mapping
• LIDAR landmark based SLAM
  • explore directions

Model Predictive Control

• explore different options:
  • MPCC Model Predictive Contouring Controller (MPCC) for Autonomous Racing
  • MPPI Model Predictive Path Integral Controller (MPPI)
• start working on simulator