Hi, thanks for your code in Conv for FF. However, there are several bugs in the code ff-cifar10-convnet.ipynb.
(1) goodness += [(h.pow(2).sum() / h.numel()).unsqueeze(0)] It's wrong. For a 1000*3*32*32 image, here the goodness is one value. But it should be 1000*1
Still, I am confused:
(2) x_direction = x / (x.norm(2, 1, keepdim=True) + 1e-4) . Do you think norm(2,1) is right? For example, for a 1D data 1000*1*784 (B*C*L), it should be norm(2,2) rather than norm(2,1).
(3) g_pos = self.forward(x_pos).pow(2).mean(dim=(1,2,3)).unsqueeze(0) why use unsqueeze(0) ? It seems to have no influence on the loss` if you do not use unsqueeze(0).
Hi, thanks for your code in Conv for FF. However, there are several bugs in the code ff-cifar10-convnet.ipynb.
(1)
goodness += [(h.pow(2).sum() / h.numel()).unsqueeze(0)]It's wrong. For a1000*3*32*32image, here the goodness is one value. But it should be 1000*1Still, I am confused:
(2)
x_direction = x / (x.norm(2, 1, keepdim=True) + 1e-4). Do you thinknorm(2,1)is right? For example, for a 1D data1000*1*784 (B*C*L), it should benorm(2,2)rather thannorm(2,1).(3)
g_pos = self.forward(x_pos).pow(2).mean(dim=(1,2,3)).unsqueeze(0)why use unsqueeze(0) ? It seems to have no influence on the loss` if you do not use unsqueeze(0).