While I agree with this, it's sadly a game theoretic reasoning, that is to say, fearful, relativistic, utilitarian.
Again, not being dismissive at all, but it says something about the state of our thinking and quality of our ideals, when we cannot make arguments on absolute grounds anymore. Or if we do, they seem quaint, passe, naive.
Everybody wants to restrict speech they dont like, but with that comes the ability for others to restrict speech you do like.
Hence, the optimal outcome is no or minimal restriction.