Trouble reproducing attack and defense example (ART black box attack using Tesseract)

[HexHexHex16]

Describe the bug

ValueError: Unrecognized input dimension. JPEG compression can only be applied to image and video data.
Regardless of whether the previous code was executed or not.

Tracecall

ValueError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_2356/2307723873.py in <module>
      2 plt.imshow(image_target)
      3 plt.show()
----> 4 print('Tesseract output is: ' + label_dict[np.argmax(classifier_def.predict(np.array([image_target], dtype=np.float32)))])

~\anaconda3\lib\site-packages\art\estimators\classification\classifier.py in replacement_function(self, *args, **kwargs)
     69 if len(args) > 0:
     70 args = tuple(lst)
---> 71 return fdict[func_name](self, *args, **kwargs)
     72
     73 replacement_function.__doc__ = fdict[func_name].__doc__

~\anaconda3\lib\site-packages\art\estimators\classification\blackbox.py in predict(self, x, batch_size, **kwargs)
    121
    122 # Apply preprocessing
--> 123 x_preprocessed, _ = self._apply_preprocessing(x, y=None, fit=False)
    124
    125 # Run predictions with batching

~\anaconda3\lib\site-packages\art\estimators\estimator.py in _apply_preprocessing(self, x, y, fit)
    303 else:
    304 if preprocess.apply_predict:
--> 305 x, y = preprocess(x, y)
    306
    307 return x, y

~\anaconda3\lib\site-packages\art\defences\preprocessor\jpeg_compression.py in __call__(self, x, y)
    117 x_ndim = x.ndim
    118 if x_ndim not in [4, 5]:
--> 119 raise ValueError(
    120 "Unrecognized input dimension. JPEG compression can only be applied to image and video data."
    121)

ValueError: Unrecognized input dimension. JPEG compression can only be applied to image and video data.

To Reproduce

1. (https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/classifier_blackbox_tesseract.ipynb)
2. In [11]:

# this is the image we want to target
plt.imshow(image_target)
plt.show()
print('Tesseract output is: ' + label_dict[np.argmax(classifier_def.predict(np.array([image_target], dtype=np.float32)))])}

Expected behavior

System information (please complete the following information):

• Windwos 10
• IDE: anaconda3 /Jupyter
• Python version 3.9.7
• ART 1.9.0
• numpy 1.20.3 / matplotlib 3.5.1/ tecsseract 5.0.1.20220118

[guiltycrazy]

I got the same problem too

[beat-buesser]

Hi @HexHexHex16 and @guiltycrazy Thank you very much for using ART raising this question! I think you are right, the recently added shape checks in JpegCompression are laerting us that the notebook uses unexpected image shapes. We have to change the shape of the arrays image_target, image_target and similar input from shape (1, 32, 106) to shape (1, 32, 106, 1). The channel dimension is missing.

For example in cell 11 we can fix it with somehting like:

print('Tesseract output is: ' + label_dict[np.argmax(classifier_def.predict(np.expand_dims(image_target, axis=[0, -1])))])

but we probably should add a global solution by extending the shape of the images in cell 2 and adapt the following cells accrodingly.

Please let me know if you would like to try to solve it, I would be happy to provide support and include it in the repo! Otherwise we'll try to fix it as soon as possible.

[HexHexHex16]

Why do assent anddissent pictures have a yellow background when uploading? I tried to eliminate the dimensional error by resaving these images with the same size, but as a result they acquired a white background and when overlaid, the text disappears on a white background. What is the peculiarity of these pictures, why their background turns yellow when loaded into the program, I still could not understand

[beat-buesser]

The colours in the images shown in the notebook are defined by the default colormap of pyplot.imshow.