Skip to content

4. Data acquisition tools

Jump to bottom
Natko Kraševac edited this page Jun 5, 2023 · 4 revisions

Intro

Training deep neural network requires large amounts of annotated data. Annotating data is often expensive in terms of time and money spent. It is also a very repetitive task which leads to human errors and noise in the labeled data. This task will guide you through the process of using the MARUS simulator for acquiring large amounts of annotated data. By now, you should be familiar with the overall usage of the simulator: modifying the scene, adding new objects, rearranging them, controlling the vehicles, adding script components and configuring the parameters. MARUS offers automatic labeling of chosen objects in camera. Dataset is stored in a commonly used format suitable for immediate training with YOLOv5 or similar architectures. Frames are stored with appropriate annotation files, split into training, validation and testing subsets. Configurable parameters include saving frequency, image resolution and ratios for training, validation and testing subsets. For image variety, we use Unity's tools for adjusting lighting, sky, clouds and fog settings. Even more realistic and complex scenery can be achieved by using commercially available assets from the Unity Asset Store.

scene_annotated

Image 1: Scene in MARUS with overlayed labels generated with annotation tool.

Camera data collection

To acquire annotated camera data from MARUS simulator follow these simple steps:

  1. Modify and configure your scene to reflect the scenario you want to capture. You can play around with modifying lighting conditions, clouds, terrain shape and texture.

  2. Place objects of interests around the scene. These can be vessels, buoys, lighthouses or any other that are needed for your use case. Note: Make sure to add Mesh Collider components to all your objects, otherwise they will not be labeled.

  3. When you are happy with your scene, you should create an empty gameObject and name it ObjectAnnotator or something similar and then attach the Camera Object Detection Saver script to it.

  4. In the inspector view, on the annotation component added in the previous step - create a new Object class and assign it a name, then in "Objects in Class" you can add different object that you want to annotate as members of that particular class. In the Image 2, two classes are added, one being Markers and other being Boats. Markers class has one "Buoy" object that has been included in the scene and the Boats class has "Sailboat" included in the scene.

  5. The camera objects used for data acquisition must be added to Camera Views list. Note: it is possible to get data from more than one camera, but be aware of the performance drop when using multiple cameras.

  6. The Camera Object Detection Saver script includes multiple parameters which you can change to get the best performance:

    • Save Frequency Hz - frequency of annotation in Hz. Note: Set this value in relation to the speed of your vehicle used for data collection. You do not want to have frequency set too high as it will generate very similar images if the vehicle is moving rather slowly.

    • Minimum Object Area - allows you to ignore small objects. It represents the minimum area of bounding box that will be logged (in pixels). You can leave the default value here.

    • Min Vertical Position- this parameter allows you to set the y-axis value (in meters) under which points of an object will be ignored. Set this to zero if your ocean level is also at zero. Note: This would ignore parts of objects that are just below the surface but still visible in the camera. Use this setting to fix this problem.

    • Vertex Step - this parameter is used when processing mesh vertices to determine which object is seen in camera. If vertex step is set to 5 - script will only check every fifth vertex. Higher number is less computationally heavy, but you lose some bounding box precision on the edges of the object.

    • Dataset Folder - full path to the folder where you want data to be saved. Default path is set to camera_detection directory in the Assets folder of the project.

    • Image Width - shows Image width, same as width resolution in Game view screen. Advice: Change Display resolution in Game view from Free Aspect to Full HD.

    • Image Height - shows Image height, same as height resolution in Game view screen. Advice: Change Display resolution in Game view from Free Aspect to Full HD.

    • Dataset can be separated into train, validation and test subsets. Relations between these three subsets can be set using Train Size, Val Size and Test Size sliders. If you wish to have just the train and validation subsets, that can be accomplished by unchecking Generate test subset checkbox.

objectAnnotator

Image 2: Camera Object Detection Saver script setup

  1. Now that you've added all the objects you want to detect inside the annotation script, you are almost ready to start collecting the dataset. Don't forget to add the boat object with one camera and a simple WASD controller to the scene. Now you can hit the Play button and start driving around your scene. Make sure to check the Enable option on the Camera Object Detection Saver component to start creating data. When you think that you have enough data, just stop the simulation and the dataset will be available in the directory you have set in the annotator component.

In the next image we have one example of MARUS project ready for annotation of the camera data:

annotator_insp

Image 3: Project ready for the annotation of the camera data

Clone this wiki locally