He's not saying removing everyone's vote, he's saying removing the vote for half of the adult population on the basis of gender, because he doesn't like how they vote.
Are you truly making the argument that disenfranchisement of groups based on demographics has as little to do with libertarianism as the results of a sports game?
The idea behind the NFL game example was to highlight the distinction between the notion of completeness with that of consistency. The connection (or lack thereof) between Libertarianism and the NFL game, or with enfranchisement, is a matter of logic, not some distance metric based on whether it's in the realm of politics.