Created a Sign Language Detector using Tensorflow, which uses pretrained model SSD MobileNet to detect different sign language words in real time.
In this phase, images were collected for training. Images captured from the webcam of laptop are purely of the developer gesturing in sign language. After capturing the images, they were resized, cropped, and labelled using an annotation tool called LabelImg.
Below is a visual example of the developer gesturing the sign 'Wrong':
For the process of Data Labelling, a graphical image annotation tool called LabelImg was used. LabelImg is a free, open-source tool for graphically labelling images. It’s written in Python and uses QT for its graphical interface. It’s an easy, free way to label a few hundred images to try out your next object detection project. Labels are used to segment out the Regions of Interest (ROIs), so that the model is able to identify the same, before the training process begins.
For every specific class, a label map was created which is used to generate training as well as testing record files. These are then used while configuring the TrainEvalPipeline.
The library used for creating this pipeline was TensorFlow Object Detection API. It is an open source framework built on top of TensorFlow that makes it easy to construct, train and deploy object detection models.
The model used in for this project is Single Shot Detector Model which is a pre-trained model (SSD) obtained through TensorFlow Model Zoo.
The model was configured to fit the dataset and to solve the problem at hand. The SSD model was trained for 10k training steps so as to ensure that the model performs with good accuracy score.
Below are some visual examples of the results obtained:
