Model not Training

[agentcap]

When I trained the network with the default parameter setting, the loss is getting saturated just after 2-3 epochs. And also the results are coming out to be random. Is there something I am missing or need to change?

Thanks

[erikalu]

Have you tried lowering the initial learning rate? e.g. to 1e-4

[agentcap]

I tried training the network with learning rate 1e-5, and the loss is ~0.234 a bit better than before.

And this is the inference code that I have used. Is this correct?

`
def test_gmn():

# ==> gpu configuration
ut.initialize_GPU(args)

# ==> set up model path and log path.
model_path, log_path = ut.set_path(args)

# ==> import library
import keras
import data_loader
import model_factory
import data_generator

# ==> get dataset information
trn_config = data_loader.get_config(args)

params = {'cg': trn_config,
          'processes': 12,
          'batch_size': args.batch_size
          }

trn_gen, val_gen = data_generator.setup_generator(**params)

# ==> load networks
gmn = model_factory.two_stream_matching_networks(trn_config, sync=False, adapt=False)
model = model_factory.two_stream_matching_networks(trn_config, sync=False, adapt=True)

# ==> attempt to load pre-trained GMN
if args.gmn_path:
    if os.path.isfile(args.gmn_path):
        gmn.load_weights(os.path.join(args.gmn_path), by_name=True)
        print('==> successfully loading the model: {}'.format(args.gmn_path))
    else:
        print("==> no checkpoint found at '{}'".format(args.gmn_path))

# ==> print model summary
model.summary()

inp_img = ut.load_data('data/image.jpg',dims=(255,255,3),pad=0)
ex_patch = ut.load_data('data/patch.jpg',dims=(63,63,3),pad=0)

inp_img = np.expand_dims(inp_img, axis=0)
ex_patch = np.expand_dims(ex_patch, axis=0)
inp_img = data_generator.preprocess_input(np.array(inp_img, dtype='float32'))
ex_patch = data_generator.preprocess_input(np.array(ex_patch, dtype='float32'))

inputs = {'image_patch': ex_patch, 'image': inp_img}

outputs = model.predict(inputs,batch_size=1)
outputs = np.squeeze(outputs,axis=0)
outputs = np.squeeze(outputs,axis=2)

im = Image.fromarray(outputs,'L')
im.save('data/out.jpg')
im.show()

test_gmn()
`

[akshitac8]

any update for the testing code?

[erikalu]

@agentcap Your code is missing a line to load the trained weights into the model before predicting, e.g. model.load_weights(args.model_path, by_name=True)

Otherwise, it looks good to me.

[akshitac8]

@agentcap How did you extracted the ex_patch from the testing image as we don't have the ground truth(dot annotations) for the testing image?

[erikalu]

@akshitac8 You should use an exemplar patch (or multiple) from the training set.

[akshitac8]

Thank you @erikalu for the details.
@agentcap were you able reproduce the MAE on CARPK Dataset?

[agentcap]

@agentcap Your code is missing a line to load the trained weights into the model before predicting, e.g. model.load_weights(args.model_path, by_name=True)

Otherwise, it looks good to me.

Hi @erikalu , in the above code snippet, there is a section under the title "attempt to load pre-trained GMN" in which we are loading the trained model. And I am passing the path to the trained model via the gmn_path argument. Can you share the trained model so that I can try to find my mistake.

Thank you @erikalu for the details.
@agentcap were you able reproduce the MAE on CARPK Dataset?

Hi @akshitac8 I was not able to reproduce the results.

[erikalu]

Hi @erikalu , in the above code snippet, there is a section under the title "attempt to load pre-trained GMN" in which we are loading the trained model. And I am passing the path to the trained model via the gmn_path argument. Can you share the trained model so that I can try to find my mistake.

Hi @agentcap, I've updated the README with the link to the pretrained weights. There is still an issue with your code, since the code snippet you mention loads the weights into the gmn object, but you are using the model object to predict. So just replace model with gmn in the prediction line.

[tuongtranngoc]

@erikalu and @agentcap,
i used pretrained model and def test_gmn() but result is very bad.
i test on Unmanned aircraft image.
you tell me why not>>
Thanks

[erikalu]

@NgTuong Here are some common issues/debugging tips:

• If summing the heatmaps to produce the count, be sure to normalize by 100, the scaling factor that was used to facilitate training (see line https://github.com/erikalu/class-agnostic-counting/blob/master/src/data_generator.py#L84)
• The pretrained model was trained for a specific exemplar patch size; make sure that your patch input is the same size (see the config file) and that the instances to count are also at roughly the same scale. If they are at multiple scales, you may want to fine-tune on your dataset with multiple patch sizes.
• Depending on your task and data, the pretrained GMN might just not be good enough, and you will have to adapt to your dataset. To check whether this might be necessary, try visualizing some of the heatmaps and checking to see if the results look reasonable. Sometimes simple thresholding and local maximums are enough to produce reasonable detections/counts.

[tuongtranngoc]

@erikalu thank you so much

[Anislar]

somme one can help me please ?

[jaideep11061982]

@erikalu can this be useful to count the occluded objects with multiple kinds. Say in retail shops cart
I dont know how we can provide a patch image to count in such case