It's always been my assumption that we wash our hands when we go to the bathroom for two reasons: 1. Wiping your butt can lead to feces getting in your hand. A lot of diseases can come from feces. 2. It's a good frequency to wash off whatever else you've been touching that day.

Assuming you go to the bathroom 3-7 times a day, that's that many opportunities to wash off the dirt you were digging in, trash you picked up off the street or someone else's hand you shook.