Hamel Husain: same model for task and evaluation is usually fine if validated

Hamel Husain, in an FAQ post on his blog, answers whether the same model can be used for both the main task and evaluation: he says "usually, yes." The condition he attaches is that the model acting as judge should be tested for agreement with human labels on a held-out test set.

The post itself is the primary source. The reported answer is a short summary, and no supporting data or examples from the full post appear in the available report.