Generally, you would only care about up-votes from people you trust, and if you vote down stuff that your friends up-voted, then your trust level in those friends would be reduced, rapidly turning down the volume on the other stuff that they promote.