> The trouble is deciding who gets to wield that force

Whoever is wielding the force is the one gets to wield it. We can have a more civilized society by strictly limiting the wield the force by limiting opportunity to weild it by clear defined individual rights.