Think of the impact if the site is blocked by the browser's red malware warning UI (Google's safe browsing or smartscreen in IE) because a compromised file was hosted on the main domain.

It is much harder to get the site whitelisted once it has been compromised since it is on the same domain.

In my job, we had a site domain that was falsely flagged as malware-infected and it took about a few days before it was confirmed as removed (it is not automated apparently). It was never made clear why it was flagged in the first place but it scared us and we had to isolate any downloadable files (installer, ads, etc) away from the main domain as much as possible.

AFAIK, Google Safe Browsing will show that warning on your site even if the malware came from a third-party domain.

But what's easier to remove; your site or the third party links on the website?

I don't remember third party domains being problematic to remove compared to the first party stuff?