There is… little disagreement on this aspect of the First/Fourth/Fifth/etc., though.
> revoking their visa is not jail
The First Amendment protects you from non-jail government consequences just fine, for obvious reasons - "we're fining you $1M for your speech" would have just as much impact.
I'm against the government jailing a visa holder for their speech, but revoking their visa is not jail.