Nathan Lambert defends the AAv5 index and open-model benchmarking

Nathan Lambert says he feels a need to defend both the AAv5 index and open models at the same time.

In an X post, he argues that benchmarking is messier, more opinionated and harder than most people give it credit for. The post does not specify the criticisms he is responding to or the evidence behind his argument.