Thanks, I now found a wiki describing these mechanics, and it turns out that UO had an entire system of markings for players who committed illegal activities. This clears things up a bit.
Idk where the ‘killing the player’ part came from—though local game journalism at that time was quite, uh, idiosyncratic. But, as the workings of these guards were somewhat hazy in my mind, they gained an ominous air.
I think I know why you have this hazy memory! Oftentimes these robber groups patrolled around the edges of the "safe zones" where it's possible to call guards. When there were battles, it was often chaotic with multiple characters moving and mauling each other. Sometimes an innocent player would be attacked near the border, they would defend themselves and accidentally do some action which marks them as a criminal (for example, moving inside the border of the safe zone and defend against an attacker who is outside the safe zone and not marked as criminal). The innocent player defending themselves might call "guards", but because they've accidentally made some particular action, the guards would treat them as the criminal and kill them.
Idk where the ‘killing the player’ part came from—though local game journalism at that time was quite, uh, idiosyncratic. But, as the workings of these guards were somewhat hazy in my mind, they gained an ominous air.