I think about this a lot in a particular context, though it has not occurred to me to express it quite in these terms.

Some people's discourse — political, or that which passes for "political" — is a torrent of bigotry. Could be one -ism in particular or multiple -isms. Whether they're -isming from ignorance, hate, or fear, and whether they do it intentionally, I usually can't say, but their statements are informationally wrong and cause harm.

So, if I examine, for example, a statement that is meant as a targeted political assault against queer/trans people, I can give a dozen reasons why it is factually incorrect, related to other bigotries, and can be found as an authoritarian tool through history. If I add: "Therefore, it is bigoted, specifically homophobic/transphobic," the person who is being criticized will drop the entire book I just wrote and say "I can't believe I was just insulted on page 357. The author is biased and hateful toward anyone who thinks differently than they do!"

Of course they say that. If they instead reacted, "Oh, gosh, I was wrong all this time, and I hurt others. I must change my ways starting today," they wouldn't be bigots anymore. Part of the accusation of bigotry rests on the observation that a person isn't open to change. The person could (if they chose) accept well-reasoned criticism as corrective guidance. Instead, they receive that criticism as an insult against them — and always as a "baseless" insult, no matter the firmness of its basis, because, if they recognized its basis, they'd have to change their lives. This resistance to argument demonstrates the bigotry of which they are accused. It continues to underscore the point. But if the other person says, "Yup, so this is homophobia/transphobia. This is what I'm talking about," the bigot accuses them of being insulting, and this is where the argument goes to die.

Consequently — because bigots do this — the attribution of an -ism to someone else does indeed have bad political optics. Someone who is committed to bigotry (whether or not they already understand it as such) will resist any attempt by others to unmask their beliefs as false and bigoted. Identifying bigotry isn't inherently fallacious, but the bigot argues that it is — because they know that, once we are open to the possibility that any statement is bigoted, the very first statements we will identify as such are theirs. Indeed they often argue that no statement can ever be identified as bigoted because we have no idea what someone else is thinking when they open their mouths, and if we can't conclusively pin states of mind to strings of words, we can't ever assess anyone else's character, right?

The bully wants the victim to engage the argument on the bully's terms, and they don't ever want to hear a normative statement against this scenario. The moment the victim says "you've been behaving badly toward me for five years, though I've asked you to stop, and I'm starting to wonder if you're doing it on purpose," the bully says "On purpose? You just insulted me! How dare you!" (And, with that deflection, of course the bully continues to do the on-purpose thing on purpose for the rest of their lives.)