up

thesis.tex

@@ -1038,21 +1038,19 @@ \section{Results}
 	\label{fig:mask-effective}
 \end{figure}
 
-\paragraph{Replication was successful.}
+\paragraph{Self-ensembling improves robustness.}
 
 Self-ensembling proves to be highly effective. In most cases, the ensemble outperforms the final layer, with only a few exceptions. Attacks targeting the ensemble itself are generally ineffective. Combining natural training with self-ensembling yields exceptional results. With natural training, the ensemble accuracy declines much more slowly as the attack strength increases compared to models without natural training.
 
 This is particularly evident in PGD attacks: without natural training, targeting the final or intermediate layers can drop accuracy close to zero. In contrast, natural training allows the ensemble's accuracy to drop only marginally under similar conditions.
 
-\paragraph{Natural training isn't always beneficial.}
-
-The robustness-accuracy trade-off is evident through natural training.
+\paragraph{Robustness vs.\ benign accuracy.}
 
 Interestingly, the impact of natural training appears to flip depending on the presence of an attack. Without an attack or under very low-opacity masks (e.g., up to 32), natural training tends to reduce model accuracy by about 10\% almost consistently. However, as opacity increases or when an attack is applied, natural training becomes highly beneficial. This phenomenon may reflect the robustness-accuracy trade-off commonly discussed in the literature. Notably, the self-ensembling technique seems unaffected by this trade-off, which is a particularly intriguing finding.
 
 \paragraph{Outlook.}
 
-Future research could explore the use of MINE~\cite{pmlr-v80-belghazi18a} and Rényi's $\alpha$-order matrix-based functional~\cite{6954500} to estimate mutual information between self-ensemble layers and track the movement of latent representations across these intermediate layers. Ideally, these finer-grained metrics, combined with simple and effective black-box attacks, could lay the groundwork for foundational research in adversarial robustness.
+Future research could explore the use of MINE~\cite{pmlr-v80-belghazi18a} and Rényi's $\alpha$-order matrix-based functional~\cite{6954500} to estimate mutual information between self-ensemble layers and track the movement of latent representations across these intermediate layers. Ideally, these finer-grained metrics, combined with simple and effective black-box attacks, could lay the groundwork for foundational research in the intersection of machine learning interpretability and adversarial robustness.