vr 005

_publications/2024-01-14-Food.md

@@ -0,0 +1,27 @@
+---
+title: "Validation report 005: Food Image Classification"
+collection: publications
+permalink: /publication/2024-01-14-Food
+excerpt: "This study evaluates the resilience of the 'nateraw/food' Visual Transformer food classification model against data manipulation attacks, using LIME and Attention Rollout for insights. The model generally withstands most transformations, but extreme photographic effects and overlaying key non-food features significantly alter its predictions. The findings highlight the model's robustness, revealing specific vulnerabilities to strategic overlays and severe photographic distortions."
+date: 2024-01-14
+venue: 'Explainable Machine Learning 2023/2024 course'
+paperurl: 'https://modeloriented.github.io/CVE-AI/files/2023_Food.pdf'
+citation: 'Tomasz Silkowski. (2024). &quot;Vulnerabilities in Food Image Classification.&quot; <i>Github: ModelOriented/CVE-AI</i>.'
+tags:
+  - Food Image Classification
+  - Visual Transformer
+  - LIME
+  - Attention Rollout
+---
+
+This study assesses the resilience of the ’nateraw/food’ Visual Transformer, a food classification model, against common data manipulation attacks. Employing methods like LIME and Attention Rollout for insight, the research finds that the model withstands most transformations, some extreme photographic effects and methods of overlaying key non-food features can significantly alter the predictions. These results highlight the model’s robustness, with implications for understanding the vulnerabilities of advanced computer vision systems.
+
+<hr/>
+
+Our analysis shows that the model is generally resilient to data manipulation. As illustrated in Figure 1, out of 16 transformations (10 photographic and 6 overlays), only 3 impacted the model's predictions significantly: 180-degree rotation, an overlay of the plate and the application of grayscale filter. These techniques vary in their destructiveness of information, however, it’s important to note that plate overlay impacted the model significantly more than its 'shadow'. This success leads us to conclude that strategically selected non-food features when overlaid on unrelated images, can effectively alter model predictions. As for photographic distortions, the impact of extreme measures, such as half-full rotation and grey filter, is consistent with loss of information in the image. This outcome demonstrates that only drastic photographic transformations to the image have a significant effect on the model’s reasoning.
+
+
+<hr/>
+
+Link to original model: [Food Classifier](https://huggingface.co/nateraw/food)
+