If I go into a private gym and start smashing things, they would want to fine me and kick me out, but it may not be worth several thousand dollars to sue me for the damages.

HOAs have inherent collateral. Indemnification is the answer. If the legal fees wrack up, you take the house. But only through the courts.