DroneZ: App for connecting the drone to the users Phone

In this project, there are currently two applications:

• Flutter application

• Andriod Stuido application

• Facial detection and tracking

• Position estimation and localization

In the future we plan to implement more features from the Andriod Stuido applaction to the Flutter application for cross platform support.

Flutter App Features:

• Connect to Tello drone
• Manaual Controls for the Tello drone
• Automatic Controls for the Tello drone where users can excute pre-programmed paths
• receieve stream data from the Tello drone

Andriod Studio App Features:

• Connect to Tello drone
• Manaual Controls for the Tello drone
• Automatic Controls for the Tello drone where users can excute pre-programmed path
• Display video stream and preform YOLO object dectection algorithm

Getting Started

• To deploy the application on your machine, you will need to install Dart and Flutter for the Flutter app, and Android Studio for Android app
• To connect to the drone, Tello drone is neeeded

Prerequisites

• Tello Drone (Model:TLW004) (if you want to fly it)
• Android Device to download our application (dronez beta app)
• Download Android Studio and Flutter according to your operating system

Installation

With the environment configured, just clone this repository. You might also need to add any packages that might not be installed.

Running Builds

For Flutter application:

• once in the cloned directory, execute:

flutter run 

For Andriod Studio application

Open android studio > navigate to cloned directory and open app_AS_only > Build > run app (preferably connecting your phone)

Notice

• video stream feature don't work on emulator devices because of connection issues between the device and the drone. If you want full functionalities please test on actual andriod devices
• For Android app, since the size of the app is big due to the object detection algorithm, it will not be able to build and install on the online emulator. Thus, actual physical phone is needed to run Android app

Connecting to drone and Running Algorithm

• Connect to the drone by clicking "wifi-figure" button
• When the battery and connection status are in green, the drone is connected
• Then navigate to the face_tracking.py and run the file using any suitable IDE (i.e. VS Code)

How to use Android App

Default page

- When the app is opened, the default page is the control selection page - Select "Manual Control" to manually control the drone with joysticks - Select "Automatic Control" to automatically control the drone with command or in pre-programmed path

Manual Control

Default Manual Control Page

Manual Control page with Live Feeding View

Connecting to drone

• Connect to the drone by clicking "wifi-figure" button
• When the battery and connection status are in green, the drone is connected

After connecting

• Able to turn on the live feeding view by clicking "video view" switch
• If the video view is on, the switch will turn in green
• Take off the drone by clicking taking off button on the right
• After taking off, able to control the drone with joysticks
• Land the drone by clicking landing button under the take off button
• When battery is lower than 15%, the battery status will turn in red

mobizen_20230504_193314.1.mp4

Automatic Control

Default Automatic Control Page

Automatic Control page with Live Feeding View

Connecting to drone

• Connect to the drone by clicking "wifi-figure" button
• When the battery and connection status are in green, the drone is connected

After connecting

• Able to turn on the live feeding view by clicking "video view" switch
• If the video view is on, the switch will turn in green
• Take off the drone by clicking "TAKEOFF" button
• Choose direction and amount of distance from the drop down boxes and click "GO" to submit distance command
• Choose rotation and amount of angle from the drop down boxes and click "GO" to submit rotation command
• "GO" buttons are enabled when either both direction and distance value or rotation and angle value are chosen; indicated with white background when enabled, otherwise in gray
• To make the drone to fly itself with pre-programmed path, click "AUTO" button

mobizen_20230504_195044.1.mp4

Facial Detection and Tracking

Installing Packages

• On terminal, make sure you have the OpenCV and DJI Tello packages installed

pip install djitellopy

• Navigate to the object_tracking/tello folder and ensure all the files are properly installed
• Make sure to turn on the drone and connect computer to drone wifi
• Run "face_tracking.py"

Notice

Before running "face_tracking.py", please place the drone away from obstacles and people. The drone will take off automatically and start detecting and tracking the closest face it detects as soon as the python file starts running.

Position estimation and localization

• Readme file is available in the position_estimation_and_localization directory