I appreciate this insight.

Recently I had a conversation with someone I know well. They were curious about my assessment that a specific public figure is "transphobic." They said, perhaps by way of correction: "That person believes that sex is binary."

Yes. People who have a focused interest in promoting the idea that "sex is binary" are making a coded statement. It's not-so-veiled. The ideology of "sex is binary" may not be precisely identical with anti-trans beliefs, but usually it operates as if it were. In the realm of pure ideas, I suppose we could tease apart the ideology of "the sex binary" on the one hand, and transphobia (anti-trans sentiment, belief, thought, action) on the other. But there's not so much difference in their real-world functionality. Laws that codify the claim that "sex is binary" are how transphobia attacks trans people, making it illegal for us to get identity documents, healthcare, etc.