Objectivity is not what you are paying for

No judge can be objective, but a good one can be fair and honest, and will tell you the difference rather than pretend otherwise. Buyers use the three words as if they mean the same thing when deciding whether to trust a judge; they do not, and only the first two are promises a person can keep.

Fair: applying the same standard you agreed to

Fair means the judge assesses what is in front of them by the standard the brief set, rather than by mood, by whether they liked your message, or by anything unrelated to the job. This is a real, checkable commitment - a judge who quotes a register and then delivers something meaner or softer than agreed has been unfair, and that is a legitimate complaint. Fairness is procedural: did the judge do the thing they said they would do, to the material they were given. It has nothing to do with whether their underlying taste matches yours, and it does not require them to have no taste.

Honest: saying what they actually think

Honest means the opinion delivered is the judge's real one, not a flattering or harsh performance chosen to please rather than to respond. A judge can be encouraging and honest at the same time - kindness in delivery is not the same as dishonesty in content, and most good judges hold both. What honesty does not mean is that the opinion is correct, or that a different honest judge would reach the same one. Two judges can both be scrupulously honest and disagree, because an opinion is a personal read, not a fact being reported. This is also why comparing two judges' honest reviews of the same material and finding they differ is not evidence either one is lying - it is exactly what should happen when two honest people give their real opinion, and expecting them to converge misunderstands what honesty was ever promising.

Objective: the one a person cannot offer

Objectivity means the same input produces the same output regardless of who is doing the assessing, and that is a property of instruments, not of people. A judge has seen some material and not other material, has tastes formed by a life that is not yours, and cannot set that aside by trying harder or promising to. Research on attractiveness judgements puts a size on it: across three experiments, Hönekopp (2006) found that "private taste is about as powerful as shared taste" in how people rate faces, so a large part of any rater's verdict is theirs alone. Even trained professionals are moved by what they know: in clinical trials with subjective outcome scales, Hróbjartsson and colleagues (2013) found "nonblinded assessors exaggerated the pooled effect size by about 68%" compared with blinded ones. Asking a judge to "just be objective" is asking them to stop being a person, which is also the reason you hired one - their subjectivity is the product, not a defect they should be overcoming. A judge who claimed true objectivity would either be lying or would have quietly turned into a rubric, and a rubric is a different, cheaper thing than what you paid for. It is also worth separating objectivity from consistency, since the two get conflated: a judge can be highly consistent in their own taste - reliably preferring the same things across many clips - without that taste being objective in any sense that would hold for a stranger reading the same material.

Where objectivity actually lives

If bias-free comparison is genuinely the goal, it exists elsewhere, built for exactly that: a model trained across a large, consistent set can return the same read from the same input every time, which is what accuracy means for that kind of system in a way it can never mean for a person. Numbers scored by a dedicated tool chase that same consistency for the same reason. And if the underlying question is really about size rather than opinion at all, a stated measurement settles it without needing anyone's judgement in the loop.

None of this makes a person's read less worth having. It means a judge should be evaluated on fairness and honesty, which are real and delivered every time a good one works, and not on objectivity, which was never on the menu. What a review is not covers the wider set of misreadings this one belongs to, and Rate Cock's guide to choosing between the two approaches is the place to go if objectivity is actually the deciding factor for you. A judge who tells you upfront that they cannot be objective, only fair and honest, is giving you more accurate information than one who claims otherwise - and it is a reasonable thing to ask them to say plainly before you send a brief.