Validation accuracy stays 0.5 when Retraining ResNet50, others are fine

[hkoelewijn]

I am comparing several architectures for retraining.
Using the Kaggle 'dogs vs cats' dataset, I set up the following:
Data generator:

train_data_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
train_data = train_data_generator.flow_from_directory(
    directory='{}/train'.format(PATH), shuffle=True, target_size=(sz, sz),
    batch_size=batch_size, class_mode='binary')

test_data_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
validation_data = test_data_generator.flow_from_directory(
    directory='{}/valid'.format(PATH), shuffle=True, batch_size=batch_size,
    target_size=(sz, sz), class_mode='binary')

Model:

pretrained_architecture = tf.keras.applications.InceptionV3(include_top=False, weights='imagenet', input_shape=(sz,sz,3))

# add a global spatial average pooling layer
x = pretrained_architecture.output
x = tf.keras.layers.Flatten()(x)

# let's add a fully-connected layer
x = tf.keras.layers.Dense(1024, activation='relu')(x)

# and final prediction layer
predictions = tf.keras.layers.Dense(1, activation='sigmoid')(x)

# this is the model we will train
model = tf.keras.Model(inputs=pretrained_architecture.input, outputs=predictions)

# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in pretrained_architecture.layers:
    layer.trainable = False

Training:

model.compile(loss='binary_crossentropy', optimizer='SGD', metrics=['accuracy'])
model.fit_generator(train_data, epochs=1, validation_data=validation_data) #, callbacks=[lr_finder])

I am using default sizes of each model with batch sizes of 32.
I ran this setup on my local machine: (Windows10/GTX970) and on Googles Colab (Tesla K80)
Using InceptionV3 and VGG19, the training and validation run as expected.
When I use Resnet50, the training looks fine, but validation returns 0.5 accuracy
A sample of some predictions after training:

array([[0.14550358],
       [0.15015373],
       [0.14434135],
       ...    
       [0.13943274],
       [0.14167683],
       [0.15093516],
       [0.14429297],
       [0.13792236]], dtype=float32)

It looks very suspicious. Also, it does not matter if I predict samples from my validation or training set, the results are similar.

[RunshengSong]

This is related to the issue of BatchNormalization in Keras, see this post:https://github.com/keras-team/keras/issues/9214.

Many workarounds are suggested in that post.

[chrike-platinum]

I also had this problem. It's not only related to the batch normalization. Don't rescale your data 1/255. But rather use the preprocess_input function in your generator.
from keras.applications.resnet50 import preprocess_input
The Keras ResNet pretrained weights are learned on a dataset with different preprocessing than Inception or VGG.

[phylliskaka]

@RunshengSong I also had this problem. My validation accuracy stuck at 51% while other network is working fine. I found the link you attached is not valid anymore. Could you describe this more detail?

[RunshengSong]

@phylliskaka Use keras version 1.2.2 solved my problem.

[A3itor]

@RunshengSong I'm facing exactly the same problem today.... what did you guys did to solve the problem?? I'm using Anaconda and I can't downgrade Keras to 1.2.2 as suggested. Also used these lines of code:

    tr_datagen = ImageDataGenerator(dtype='float32',preprocessing_function=preprocess_input)
    val_datagen = ImageDataGenerator(dtype='float32',preprocessing_function=preprocess_input)

instead of:

    tr_datagen = ImageDataGenerator(rescale=1./255, zoom_range=0.3, rotation_range=50,
                                    width_shift_range=0.2,height_shift_range=0.2,
                                    shear_range=0.2, horizontal_flip=True, fill_mode='nearest')

    val_datagen = ImageDataGenerator(rescale=1./255)

I haven't seen any more posts about this issue on google or github....
Any help is really appreciated

[Sanket758]

I also had this problem. It's not only related to the batch normalization. Don't rescale your data 1/255. But rather use the preprocess_input function in your generator.
from keras.applications.resnet50 import preprocess_input
The Keras ResNet pretrained weights are learned on a dataset with different preprocessing than Inception or VGG.

Thanks to this, i resolved my issue. my model got 90% val_acc under 10 epochs. Really appreciate the help.
Though if some is still having trouble, even after doing this, i suggest training your model separately and add it to the top of ResNet50. And retrain them together.

[MW0982]

I also had this problem. It's not only related to the batch normalization. Don't rescale your data 1/255. But rather use the preprocess_input function in your generator.
from keras.applications.resnet50 import preprocess_input
The Keras ResNet pretrained weights are learned on a dataset with different preprocessing than Inception or VGG.

You are a true hero!

[engsaeda]

@chrike-platinum
how I can use this from keras.applications.resnet50 import preprocess_input

[sasanradan1999]

@engsaeda

add the preprocessing_function=preprocess_input to your imageDataGenerator like the example below

from keras.applications.resnet50 import preprocess_input
image_generator = imageDataGenerator(preprocessing_function=preprocess_input)

[aryatomarAI]

My model's accuracy stuck around 0.5 even after doing this correction:
image_generator = imageDataGenerator(preprocessing_function=preprocess_input)
I have also tried different optimizers but my model's accuracy is not increasing

[tavva-srinivas]

I also had this problem. It's not only related to the batch normalization. Don't rescale your data 1/255. But rather use the preprocess_input function in your generator. from keras.applications.resnet50 import preprocess_input The Keras ResNet pretrained weights are learned on a dataset with different preprocessing than Inception or VGG.

Thanks man