Similarity metrics are limited for evaluating LLM outputs

Hamel Husain argues that similar wording does not show whether an LLM answer works for a specific application, so teams should check concrete failure modes. He adds that similarity metrics can still help with retrieval evaluation and measuring output diversity.