Author: Bartosz Ptak

If you use a dataset or model, please inform about the source.

Install requirements

conda create --name gaze python=3.6.4 
conda activate gaze

conda install -c conda-forge dlib
conda install jupyter tensorflow-gpu=1.12

pip install \
  numpy==1.16 \
  requests \
  imutils \
  opencv-python==3.4.1.15 \
  pandas

Downloads

Use python get_big_files.py to download all files
or
download directly:

• shape_predictor_68_face_landmarks.dat
• Eye dataset
• Train model and logs
• Inference model

Dataset

Dataset consists of 2670 training data and 256 test and validation data. Data structure:

• image file with resolution 120x60px:
  
• image landmarks:
  

file,L_x,L_y,R_x,R_y,CC_x,CC_y,CL_x,CL_y,CR_x,CR_y,CU_x,CU_y,CD_x,CD_y
test_imgs/eye_8.png,14,32,113,41,89,33,72,33,105,38,89,16,90,49

Model

Model based on hg_blocks.py from Stacked Hourglass Networks for Human Pose Estimation paper.

Train

• loss (MSE multiplied by 4 because heatmaps are scaled + punishment for the wrong side from the inside of the eye)
  
• metric (MAE multiplied by 4 because heatmaps are scaled)
  

Sample

1. Input image:
   

2. Result heatmaps

3. Result points