Well that's an inherent problem of having multiple people with custody of a single child.
Ideally a compromise can be reached, but in extreme cases I suppose it could end up with litigation. But still, this is a private dispute, not something that should require outsourcing parenting to the government.
I'll do my best raising my kids well, as you said.