HandTracking: a deep learning project

Our objective is to train a deep neural network able to recognize hands' movement and provide a real time 3D recostruction of it.

Roadmap

• Acquire data
• Find hands in a picture
• The pure deep learning step
• The reconstruction

Acquire data. A lot of data.

The first step of each machine learning project is to find a data set. This is the aim of this website: we shot several videos with a special camera in order to have many frames (i.e. the pictures proposed in this website) to be labeled. In our case the labels are all the junctions of the hand(s) showed in the pictures. These are required in order to train our deep neural network.

Find hands in a picture. Bounding boxes.

To train the neural network we need to focus the attention on the area of each picture where hands are present. So we need to compute the bounding box of each sample picture.

The pure deep learning step. Training and prediction.

Finally we can train our neural network with the huge amount of data collected.

The reconstruction.

In the final step we plan to show a 3D real-time reconstruction of hands movement.

---

## About us We are Computer Science students at Politecnico di Milano (♥) working on this project for a Deep Learning course. Luca Cavalli, Gianpaolo Di Pietro, Michele Bertoni, Matteo Biasielli, Mattia Di Fatta