F1-score loss for RRN

[MjdMahasneh]

I am trying to implement F1-score loss for the RPN's classification head, but the loss is stuck at ~0.9993782699 during training. here is my implementation:

def rpn_loss_cls_F_one(num_anchors):
        def rpn_loss_cls(y_true, y_pred):
                if K.image_dim_ordering() == 'tf':                        

                        tp = K.sum(K.cast( y_true[:, :, :, :num_anchors] * y_pred[:, :, :, :], 'float'), axis=0)
                        fp = K.sum(K.cast((1-y_true[:, :, :, :num_anchors])* y_pred[:, :, :, :], 'float'), axis=0)
                        fn = K.sum(K.cast(y_true[:, :, :, :num_anchors]*(1- y_pred[:, :, :, :]), 'float'), axis=0)

                        p = (tp / (tp + fp + K.epsilon()))
                        r = (tp / (tp + fn + K.epsilon())) 

                        f1 = 2*p*r / (p+r+K.epsilon())
                        f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)
                        return 1 - K.mean(f1)


        return rpn_loss_cls

Is my implementation correct? if so, what possible reasons can I further look into to explain this strange loss behaviour during the training?

Thanks :)