Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paper's schedules are inconsistent with code #5

Open
loris2222 opened this issue Aug 3, 2021 · 2 comments
Open

Paper's schedules are inconsistent with code #5

loris2222 opened this issue Aug 3, 2021 · 2 comments

Comments

@loris2222
Copy link

Hi, I am still having issues replicating your results by following the instructions and algorithms in the paper.

I have found that the schedules described in the paper are not consistent with what is present in the code, and I think that schedule optimization is a very relevant issue for performance. Indeed, there is high variability of fooling rates between schedules.

As an example, this is the schedule for Patch-RS as described in the paper (page 15):
image

And this is what I have found in the code (rs_attack.py 200):
image

@fra31
Copy link
Owner

fra31 commented Aug 3, 2021

Hi,

the schedule in the code should be the correct one, I think the one in the text wasn't changed from the previous version of the paper. Thanks for noticing it, I'll make sure to update it. Note however that, with the parameters we use, the size of the updates becomes 1x1 (so cannot decrease) at iteration 2000, so the remaining reductions do not change the updates. Also, at iteration 6000 (for untargeted image-specific patches) the single-channel updates start.

Which kind of differences in the results compared to what reported do you get?

@loris2222
Copy link
Author

Ok, thank you for the heads up.

Well I have been long struggling as I cannot manage to replicate your results with my code, even by following your directions as detailed in the paper. I would have used your code but it is very hard to understand and modify for my purposes, so I am going through it trying to decipher the functioning, but it seems to me like there is some piece missing.
I am not sure whether it's in the optimizer but I think that is my best guess. I will continue working on it and maybe I'll find it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants