Make QUE use untrusted data explicitly #45
Conversation
|
I checked that the tests still run (with some minor changes) and that Mahalanobis still gives the same result on one task (couldn't easily do the same for QUE since I haven't used that detector much myself). |
There was a problem hiding this comment.
I agree that this makes more sense now. I mostly really like this (makes it stable to inference time choices like batch_size and poison ratio which it wasn't before). On the other hand, it does become a different type of detector, but I guess comparisons are more transparent this way.
Looks great overall and I approve. For comparing against previous results, the difference is big enough that I don't expect old and new numbers to be very close. I still expect it to beat Mahalanobis as long as untrusted data is from same distribution at training and inference time.
I'll see if I'll find the time to check out current QUE performance. Otherwise, if you run it and results aren't much worse than Mahalanobis, then I'd say it is fine to merge.
|
Closing in favor of #53 (which includes these changes but fixes conflicts and should be merged soon). I checked QUE performance on MNIST corner pixel backdoors there and it seems fine, about as good as Mahalanobis (though I didn't directly compare against older numbers) |
Previously, QUE relied on getting large batches of data at a time to compute statistics on untrusted data. That was a deliberate design decision at one point before we had the current task structure, but I now think it's pretty bad. We should make all detectors that want untrusted data explicitly require untrusted data during training.
To support that, I had to make it possible for a statistical detector to use both trusted and untrusted data. This added some complexity, maybe there's a good way to redesign it, but I'm ok with this version for now (at some point, we'll likely want to refactor statistical detectors quite a bit anyway, see #42 ).
Ideally, we'd somehow enforce that detectors treat the batch dimension as an actual batch dimension, but the only way I see would be to have them implement elementwise anomaly scores, which we'd then need to compile to make it efficient. So I think we should just keep that in mind manually.