GitHub - interactivetech/Hand-Sign-Recognition-CoreML: Implements end-to-end training, error analysis, and deployment of Hand Sign Recognitino to webcam (opencv) and mobile (CoreML)

interactivetech / Hand-Sign-Recognition-CoreML Public

Notifications You must be signed in to change notification settings
Fork 1
Star 7

Implements end-to-end training, error analysis, and deployment of Hand Sign Recognitino to webcam (opencv) and mobile (CoreML)

7 stars 1 fork Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
224x224_logs		224x224_logs
cs230-code-examples-master		cs230-code-examples-master
data		data
experiment/test		experiment/test
logs		logs
model		model
tf-keras-tutorial-master		tf-keras-tutorial-master
.gitignore		.gitignore
CoreML_AVKit_test.zip		CoreML_AVKit_test.zip
Hand_Sign_Classification_mobilenet_test.mov		Hand_Sign_Classification_mobilenet_test.mov
Hand_Sign_Recognition.mlmodel		Hand_Sign_Recognition.mlmodel
Hand_Sign_Recognition_3.mlmodel		Hand_Sign_Recognition_3.mlmodel
README		README
Test_Mobilenet.mlmodel		Test_Mobilenet.mlmodel
WebcamVideoStream.py		WebcamVideoStream.py
__init__.py		__init__.py
build_dataset.py		build_dataset.py
common.py		common.py
convert_to_coreml.py		convert_to_coreml.py
evaluate.py		evaluate.py
incorrect_label_function_test.py		incorrect_label_function_test.py
mobilenet_issue_conversion_test.py		mobilenet_issue_conversion_test.py
test_model_webcam.py		test_model_webcam.py
test_tf_keras.py		test_tf_keras.py
train.py		train.py
video.py		video.py
video_threaded.py		video_threaded.py
webcam_demo.py		webcam_demo.py

Repository files navigation

# Project that implements end-to-end training, error analysis, and deployment to webcam (opencv) and mobile (CoreML)

Final CoreML model is Hand_Sign_Recognition_3.mlmodel. This model when training achieved 94% accuracy on the test set

CoreML_AVKit_test.zip has the xcode project to run the CoreML model on a live iphone feed.

Video of the result in action can be seen here: https://www.youtube.com/watch?v=Tl7qRWQFqJc&feature=youtu.be

Dataset is the SIGNS dataset developed by the CS230 class. The recognition challenge is to recognize the number of fingers showing of one hand in an image.

Main features include logging, training and modifying hyper-parameters using a json file, and logging performance and misclassification with Tensorflow. Code also includes how to run on threaded webcam using opencv and deploying to CoreML for iOS. 

This project was inspired by CS230 project repository and Andrew Ng's Structuring Machine Learning Project videos.
Copyright @Andrew Mendez 2018