BrendelBethgeAttack Assertion Error

[LGB7]

Hello, I am having another problem while trying to generate adversarial examples using the BrendelBethgeAttack. After running for quite a while, I am getting the error attached below, and I haven't found any solution yet to solve the error.

Note that the code worked for a sample of 1000 elements from the dataset but isn't working for the whole dataset.

Here is a sample of the code I am running:

import torch
import torchvision
from cifar10_models.vgg import vgg11_bn

device = 'cuda' if torch.cuda.is_available() else 'cpu'

#Pretrained model
model = vgg11_bn(pretrained=True).to(device)
model.eval()

mean=(0.4914, 0.4822, 0.4465)
std=(0.2471, 0.2435, 0.2616)

import logging
from art.attacks.evasion import BrendelBethgeAttack
from art.estimators.classification import PyTorchClassifier
from art.utils import load_dataset
import numpy as np

#Configure a logger to capture ART outputs
logger = logging.getLogger()
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter("[%(levelname)s] %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

(x_train, y_train), (x_test, y_test), min_pixel_value, max_pixel_value = load_dataset('cifar10')

x_test_norm=torch.from_numpy(x_test)

mean2=torch.from_numpy(np.asarray(mean))
std2=torch.from_numpy(np.asarray(std))

x_test_norm.sub_(mean2).div_(std2)
x_test_norm = x_test_norm.cpu().detach().numpy()
x_test_norm = np.transpose(x_test_norm, (0, 3, 1, 2)).astype(np.float32)
x_test_norm=torch.from_numpy(x_test_norm)

#Define the loss function and the optimizer
criterion = torch.nn.HingeEmbeddingLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

#Create the ART classifier
classifier = PyTorchClassifier(
model=model,
clip_values=(min_pixel_value, max_pixel_value),
loss=criterion,
optimizer=optimizer,
input_shape=(3, 32, 32),
nb_classes=10,
)

logger = logging.getLogger()
logger.setLevel(logging.WARNING)
#Crafting the adversarial example with Brendel_Bethge
logger.warning("Create Brendel_Bethge attack")
adv_crafter = BrendelBethgeAttack(classifier)

logger.warning("Craft attack test examples")
x_test_adv = adv_crafter.generate(x_test_norm)
x_test_adv=torch.from_numpy(x_test_adv)

y_test_adv=torch.from_numpy(y_test)

[beat-buesser]

Hi @LGB7 Are you still observing this error message?

[LGB7]

Hello @beat-buesser,
Thank you again for your reply.
I have tried to run the code again with the changes you mentioned but I am still getting the same error message.

[beat-buesser]

Hi @LGB7 I'm not able to reproduce the error. Can you share your models?

[LGB7]

Hi @beat-buesser
Yes I am using the models from the following open source: https://zenodo.org/record/4431043#.YTY36ogzZPY

[LGB7]

Hi @beat-buesser
I just wanted to ask you regarding the previous remark that you mentioned for the preprocessing, I tried it on a different attack and it did change my results, but I am not quite sure why. Could you explain what the difference would be if the preprocessing was done before passing the images to the classifier or if the mean2 and std2 were passed to PytorchClassifier as preprocessing=(mean2,std2)?
Thank you for your help

[beat-buesser]

Hi @LGB7
It is important to keep input data (e.g. x) and input related estimator (e.g. clip_values) and attack arguments (e.g. eps, eps_step ) in the same scale/range.
The best is to provide unmodified x, for this example in range [0, 1], with would require clip_values=(0, 1) and, for example, attack budget eps_1 in the scale of [0, 1]. If x is in range [0, 255] then clip_values=(0, 255) and eps should be relative to range [0, 255] like eps_255=eps_1*255.
If x is provided already normalised the user also has to provide also normalised values for all other input related arguments, which is less convenient because ART can take care of it automatically if preprocessing=(mean2,std2) are provided.