Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OutOfRangeError: #1

Open
xjtushujun opened this issue Jul 15, 2019 · 3 comments
Open

OutOfRangeError: #1

xjtushujun opened this issue Jul 15, 2019 · 3 comments

Comments

@xjtushujun
Copy link

It appears that:

Traceback (most recent call last):
File "main.py", line 110, in
main()
File "main.py", line 106, in main
selfie(gpu_id, input_reader, model_name, total_epochs, batch_size, lr_boundaries, lr_values, optimizer, noise_rate, noise_type, warm_up, threshold, queue_size, restart=restart, log_dir=log_dir)
File "/media/shujun/sj/project/SELFIE/SELFIE/algorithm/selfie.py", line 154, in selfie
train_batch_patcher.bulk_load_in_memory(sess, train_ids, train_images, train_labels)
File "/media/shujun/sj/project/SELFIE/SELFIE/reader/batch_patcher.py", line 40, in bulk_load_in_memory
mini_ids, mini_images, mini_labels = sess.run([ids, images, labels])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_0_shuffle_batch/fifo_queue' is closed and has insufficient elements (requested 128, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/fifo_queue, shuffle_batch/n/_147)]]
@songhwanjun
Copy link

songhwanjun commented Jul 19, 2019 •
edited
Loading

I think the problem was caused by the wrong path of input directory. Please check it and let me know. Thanks.

@xjtushujun
Copy link
Author

Now read following files.
['/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/data_batch_1.bin', '/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/data_batch_2.bin', '/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/data_batch_3.bin', '/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/data_batch_4.bin', '/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/data_batch_5.bin']
Filling queue with 20000 data before starting to train. This will take a few minutes.
Now read following files.
['/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/test_batch.bin']

It seems to read the data correctly.

@songhwanjun
Copy link

songhwanjun commented Jul 31, 2019 •
edited
Loading

When I tried to run again, it was work well as follow logs:

Now read following files.
['/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/data_batch_1.bin', '/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/data_batch_2.bin', '/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/data_batch_3.bin', '/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/data_batch_4.bin', '/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/data_batch_5.bin']
Filling queue with 20000 data before starting to train. This will take a few minutes.
Now read following files.
['/data/home/songhwanjun/testing/SELFIE/dataset/CIFAR-10/test_batch.bin']
Filling queue with 4000 data before starting to train. This will take a few minutes.
[0731 12:50:52 @registry.py:121] DenseNet/conv0 input: [None, 32, 32, 3]
[0731 12:50:52 @registry.py:129] DenseNet/conv0 output: [None, 32, 32, 16]
[0731 12:50:52 @registry.py:121] DenseNet/block1/dense_layer.0/conv1 input: [None, 32, 32, 16]
[0731 12:50:52 @registry.py:129] DenseNet/block1/dense_layer.0/conv1 output: [None, 32, 32, 12]
[0731 12:50:52 @registry.py:121] DenseNet/block1/dense_layer.1/conv1 input: [None, 32, 32, 28]
[0731 12:50:52 @registry.py:129] DenseNet/block1/dense_layer.1/conv1 output: [None, 32, 32, 12]
[0731 12:50:52 @registry.py:121] DenseNet/block1/dense_layer.2/conv1 input: [None, 32, 32, 40]
[0731 12:50:52 @registry.py:129] DenseNet/block1/dense_layer.2/conv1 output: [None, 32, 32, 12]
[0731 12:50:52 @registry.py:121] DenseNet/block1/dense_layer.3/conv1 input: [None, 32, 32, 52]
[0731 12:50:52 @registry.py:129] DenseNet/block1/dense_layer.3/conv1 output: [None, 32, 32, 12]
[0731 12:50:52 @registry.py:121] DenseNet/block1/dense_layer.4/conv1 input: [None, 32, 32, 64]
[0731 12:50:52 @registry.py:129] DenseNet/block1/dense_layer.4/conv1 output: [None, 32, 32, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block1/dense_layer.5/conv1 input: [None, 32, 32, 76]
[0731 12:50:53 @registry.py:129] DenseNet/block1/dense_layer.5/conv1 output: [None, 32, 32, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block1/dense_layer.6/conv1 input: [None, 32, 32, 88]
[0731 12:50:53 @registry.py:129] DenseNet/block1/dense_layer.6/conv1 output: [None, 32, 32, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block1/transition1/conv1 input: [None, 32, 32, 100]
[0731 12:50:53 @registry.py:129] DenseNet/block1/transition1/conv1 output: [None, 32, 32, 100]
[0731 12:50:53 @registry.py:121] DenseNet/block1/transition1/pool input: [None, 32, 32, 100]
[0731 12:50:53 @registry.py:129] DenseNet/block1/transition1/pool output: [None, 16, 16, 100]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.0/conv1 input: [None, 16, 16, 100]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.0/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.1/conv1 input: [None, 16, 16, 112]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.1/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.2/conv1 input: [None, 16, 16, 124]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.2/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.3/conv1 input: [None, 16, 16, 136]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.3/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.4/conv1 input: [None, 16, 16, 148]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.4/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.5/conv1 input: [None, 16, 16, 160]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.5/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/dense_layer.6/conv1 input: [None, 16, 16, 172]
[0731 12:50:53 @registry.py:129] DenseNet/block2/dense_layer.6/conv1 output: [None, 16, 16, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block2/transition2/conv1 input: [None, 16, 16, 184]
[0731 12:50:53 @registry.py:129] DenseNet/block2/transition2/conv1 output: [None, 16, 16, 184]
[0731 12:50:53 @registry.py:121] DenseNet/block2/transition2/pool input: [None, 16, 16, 184]
[0731 12:50:53 @registry.py:129] DenseNet/block2/transition2/pool output: [None, 8, 8, 184]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.0/conv1 input: [None, 8, 8, 184]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.0/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.1/conv1 input: [None, 8, 8, 196]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.1/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.2/conv1 input: [None, 8, 8, 208]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.2/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.3/conv1 input: [None, 8, 8, 220]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.3/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.4/conv1 input: [None, 8, 8, 232]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.4/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.5/conv1 input: [None, 8, 8, 244]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.5/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/block3/dense_layer.6/conv1 input: [None, 8, 8, 256]
[0731 12:50:53 @registry.py:129] DenseNet/block3/dense_layer.6/conv1 output: [None, 8, 8, 12]
[0731 12:50:53 @registry.py:121] DenseNet/gap input: [None, 8, 8, 268]
[0731 12:50:53 @registry.py:129] DenseNet/gap output: [None, 268]
[0731 12:50:53 @registry.py:121] DenseNet/linear input: [None, 268]
[0731 12:50:53 @registry.py:129] DenseNet/linear output: [None, 10]
'# of samples: 50000'
'# of samples: 10000'
Noise Injection: pair
history length : 15
Restart: 0
1 , 0.1 , 3.9554359089687963 , 0.2579323849104859 , 1.7421533110775524 , 0.3686708860759494

I think that CIFAR-10 data did not exist in your path: '/media/shujun/sj/project/SELFIE/SELFIE/dataset/CIFAR-10/'.
Please download "data_batch_x.bin and test_batch.bin and locate it inside the path.
(you can download the dataset by clicking the link in my github.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@songhwanjun @xjtushujun