How avoid warning for binary files?

[git1106]

To make git treat a file as binary and don't show any diffs one unsets the diff attribute.

However, if I use something like this in .gitattributes

*.secret filter=git-crypt -diff

git-crypt status will complain every time

*** WARNING: diff=git-crypt attribute not set ***

How can the warning be avoided? Of course it's only a warning and one could ignore it. But routinely ignoring warnings is not a good idea, next time it might have been a serious one like

*** WARNING: staged/committed version is NOT ENCRYPTED! ***