Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with multiple FI in layer_output mode #8

Open
uditagarwal97 opened this issue Oct 22, 2021 · 0 comments
Open

Error with multiple FI in layer_output mode #8

uditagarwal97 opened this issue Oct 22, 2021 · 0 comments

Comments

@uditagarwal97
Copy link
Collaborator

Here's the error log:

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 -c 'import keras; print(keras.__version__)'
2.4.3
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 cnn-mnist.py ./confFiles/sample.yaml ./result/ 100 1 > /dev/null
2021-10-22 18:27:29.528464: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
/home/uditg/UBC/TensorFI2/src/config.py:13: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  fiConf = yaml.load(fiConfs)
Traceback (most recent call last):
  File "cnn-mnist.py", line 79, in <module>
    res = tfi.inject(model=model, x_test=test_images[i:i+1], confFile=conf)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 46, in __init__
    self.val = fiFunc(model, fiConf, **kwargs)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 316, in layer_outputs
    pred = get_pred([fiLayerOutputs])
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3792, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1605, in __call__
    return self._call_impl(args, kwargs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1645, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 598, in call
    ctx=ctx)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError:  In[0] is not a matrix. Instead it has shape [1,12,12,32]
         [[node dense/BiasAdd (defined at /home/uditg/UBC/TensorFI2/src/tensorfi2.py:315) ]] [Op:__inference_keras_scratch_graph_1104]

Function call stack:
keras_scratch_graph

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ cat ./confFiles/sample.yaml 
Target: layer_outputs
Mode: multiple
Type: bitflips
Amount: 10
Bit: N
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ 

System config:

(tfi2) $ python3 -c 'import tensorflow as tf; print(tf.__version__)'                                                                    
2.2.0                                                                                                                                                                                                             
(tfi2) $ python3 -c 'import keras; print(keras.__version__)'                                                                            
2.4.3                                                                                                                                                                                                             
$ python --version                                                                                                               
Python 3.7.10                                                                                                                                                                                                     
$ python3 --version                                                                                                          
Python 3.7.10                                                                                                                                                                                                     
$ uname -a                                                                                                                    
Linux 2403c486b851 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux                                                                                               
$ cat /etc/os-release                                                                                                    
NAME="Ubuntu"                                                                                                                                                                                                     
VERSION="18.04.5 LTS (Bionic Beaver)"                                                                                                                                                                                                                                                                                                                                                                               
PRETTY_NAME="Ubuntu 18.04.5 LTS"                                                                                                                                                                                  
VERSION_ID="18.04"                                                                                                                                                                                                
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant