“Fair” is a complex moral question that llms are not qualified to answer, since they have no morals or empathy, and aren’t answering here.

Instead they are being “consistent” and the humans are not. Consistency has no moral component and llms are at least theoretically well suited to being consistent (model temperature choices aside)

Fairness and consistency are two different things, and you definitely want your justice system to target fairness above consistency.