Hamel Husain's FAQ on AI evals: error analysis, judges, and trace review

Hamel Husain and Shreya Shankar's FAQ explains AI evals as tests of whether an AI system does what users and the business want. It recommends starting with error analysis on at least 30 traces, then turning recurring failures into binary code-based checks or LLM judges validated against human labels.