I second the idea of a controversy indicator - it's a significant piece of information that gets lost whenever the bandwagon jumps on, even with scores displayed.

I mostly see this in the form of low quality or offensive comments that get lots of up-votes from people who agree with the poster's opinion - usually if a good comment gets down-voted because the majority disagrees, someone says "why the down-votes? its' a legitimate point", and it gets boosted back to positive.