GradCAM Heatmap Inverse?

[Boraly]

Hey,

I trained a model to classify polyps. Now I tried to visualize using GradCAM. I copied the code from the tutorial, but I think the heatmap is inverted. I will add a example Image.

My Code:

``directory = '/content/polyp_paris_export/images/'
fnames = list(fname for fname in listdir(directory) if fname.endswith('.png'))
cfg = AutoTransform.from_name('efficientnet_b0')
norm = cfg.transforms[-1]
postprocessing = Normalize(-norm.mean / norm.std, (1.0 / norm.std))
model.eval()
for i in fnames:
img = directory + i
img_1 = Image.open(img)

batch_1 = cfg(img_1).unsqueeze(0)

plt.imshow(img_1)

sa_1 = model.interpret(batch_1, using=SaliencyMap()).show()

grad_1 = model.interpret(batch_1, using=GradCam(), postprocessing=postprocessing).show()
plt.show()
sleep(1)
clear_output(wait=True)
input("Press Enter to continue...")``

You can cleary see, that the SaliencyMap recognize the polyps, but the GradCAM just focused on the area around them.

I tried to just invert the Image by using 1-img. But the results doesn't look clear.

How can I fix that?

[FrancescoSaverioZuppichini]

Hey @Boraly

Very cool project, I don't know actually the GradCam has always worked for me, I was working on an integration with torchcam[https://github.com/frgfm/torch-cam] but it's not there yet. I would recommend to try with the GradCam there and seeing if there is a problem.

When you create a cam using it you can all pass a reference to the layer, that should make using glasses super easy

from torchcam.methods import GradCAM

cam_extractor = GradCAM(model, [model.encoder])

Let me know, feel free to create a jupyter-notebook sharing your code/issues