The OP's example is pure "social justice warrior" behavior.

The prejudices of others create for me no legal duty nor a moral obligation.

I was once harassed by an elderly cleaner in a Lakewood, Ohio McDonald's for wearing an NRA ballcap. Did I have some mysterious duty to remove that cap when prompted by somebody who also expressed his opinion that he "wasn't so sure that the Holocaust was a BAD thing"?