Metrics don't show up in training job

[CEP-reperio]

Hi,

I am training a model in a notebook instance on sagemaker that outputs for each epoch the following:

Epoch 1/100
loss: 0.5203468 accuracy: 0.7339375
val_loss: 0.5155295729637146 val_acc: 0.7368293501048218

I have defined a tensorflow estimator in the following way:

from sagemaker.tensorflow import TensorFlow
instance_type='ml.m5.xlarge' #'ml.m5.xlarge' #ml.p3.2xlarge'#'local'

metric_definitions = [{'Name': 'validation:accuracy', 'Regex': 'val_acc: ([0-9\\.]+)'},
                      {'Name': 'validation:loss', 'Regex': 'val_loss: ([0-9\\.]+)'},
                      {'Name': 'training:accuracy', 'Regex': 'accuracy: ([0-9\\.]+)'},
                      {'Name': 'training:loss', 'Regex': 'loss: ([0-9\\.]+)'}
                     ]

# Create the Tensorflow estimator using a Tensorflow 2.1 container
# https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/sagemaker.tensorflow.html

estimator = TensorFlow(entry_point='modules/rnn_train.py',
                       source_dir=".",
                       dependencies=['config'],
                       role=role,
                       instance_count=1,
                       instance_type=instance_type,
                       sagemaker_session = sagemaker_session, #?
                       framework_version='2.1.0',
                       py_version='py3',
                       output_path=output_data_uri,
                       code_location=output_data_uri, # ?
                       # base_job_name='tf-rnn', # ?
                       # checkpoint_s3_uri = ckpt_data_uri,  #?
                       script_mode= True,
                       metric_definitions = metric_definitions, 
                       enable_sagemaker_metrics=True,
                       hyperparameters={
                        'data_file': 'rnn_data_relative.p',
                        'epochs': 100,
                        'labels': 'onscotoma',
                        'network': 'bidir_rnn',
                        'local': False
                       })

The training runs without problems but the metric_definitions do not appear in the monitor section of the training job. Do you know how to solve this issue?

Thanks!

[CEP-reperio]

Sorry, it was just a bug on my side!