Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault (core dumped) #20

Open
fee0819 opened this issue Oct 29, 2019 · 10 comments
Open

Segmentation fault (core dumped) #20

fee0819 opened this issue Oct 29, 2019 · 10 comments

Comments

@fee0819
Copy link

fee0819 commented Oct 29, 2019

Hi, I am Asad , You give me the instructions how to run faster_rcnn_inception_v2_boat pre-trained for boat detection. After the changes as per the instructions we are facing this error.

trt

@jkjung-avt
Copy link
Owner

Please check if this is due to "Out Of Memory" (OOM). You could run sudo tegrastats in another console and look at RAM usage.

@fee0819
Copy link
Author

fee0819 commented Oct 29, 2019

@jkjung-avt Sir this is the output of this command and I don't think it is out of memory.
Screenshot from 2019-10-29 12-10-51

@jkjung-avt
Copy link
Owner

Which version of tensorflow did you use to "train" the faster_rcnn_inception_v2_boat model?

@fee0819
Copy link
Author

fee0819 commented Oct 30, 2019

sir,the tensorflow version used for training is 1.12.0 and in tx2 we are using the tensorflow version 1.11.0. I can run the ssd_mobilenet_v1_coco demo model on tx2 , but we can't use our model.

@jkjung-avt
Copy link
Owner

At this point, I'm not sure what the problem might be. Would you be able to post your pipeline config file ('faster_rcnn_inception_v2_boat.config')? I'll check whether I could spot anything suspicious.

@fee0819
Copy link
Author

fee0819 commented Oct 30, 2019

We failed to run this model but we tried ssd_resnet50_v1_boat model and it can run sucessfully but fps is very low (4 fps). We do not know what is wrong with the faster_rcnn model. Here is our pipeline config file for faster_rcnn_inception_v2_boat

Note: To build the model we give label_map.pbtxt and model in the argument. This is the config that we use to train the mode and use as it is now.

faster_rcnn_inception_v2_boat.txt

@jkjung-avt
Copy link
Owner

Please refer to my modified version of faster_rcnn_inception_v2_egohands.config: https://github.com/jkjung-avt/tf_trt_models/blob/master/data/faster_rcnn_inception_v2_egohands.config

Try making the following 2 modifications to your config file:

  1. lines 16~20: use 'fixed_shape_resizer' for inferencing instead of 'keep_aspect_ratio_resizer'
  2. line 50: reduce 'first_stage_max_proposals' to 32

You don't need to re-train the model with this modified config. Just use it to build TF-TRT model. Let me know if that works.

@MuhammadAsadJaved
Copy link

MuhammadAsadJaved commented Oct 31, 2019

Running after modifications as per your instructions throw following error. I also tried different combinations first_stage_max_proposals and detection per class etc. To make sure about changes in .config file please also check the attached modified .config file.

trtN

faster_rcnn_inception_v2_boat.txt

@jkjung-avt
Copy link
Owner

Please do specify a height and width in your 'fixed_shape_resizer'. If you don't like the numbers of (576, 1024), you could modify it to, say, (720, 1280). This setting does affect the smallest objects (boats) that could be detected by the TF-TRT model, though.

      fixed_shape_resizer {
        height: 576
        width:  1024
      }

@MuhammadAsadJaved
Copy link

Sir same error. I can run ssd_mobilenet_v2_boat , ssd_mobilenet_v1_boat and ssd_resnet50_v1_boat without any error with the same config i used for training. So I think there is something wrong only with this model during the optimization process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants