How does staying oneself have anything to do with what other people do?

(I'm failing to see the how we get to "in effect say they don't have a right to their own place". My understanding of the leftist position is that everyone has a right to their own place.)

(I'm all in favour of people's freedom to swing their fists about, as long as they're avoiding other peoples' faces)