AA-Omniscience v1.1 has a new public set on @huggingface calibrated to match the difficulty of the private set. At release, public and private scores have a 0.99 correlation for AA-Omniscience Net Score, with a median difference of 0.0 points. Accuracy and hallucination rate also have 0.99 correlations, with median differences of -0.07 and -0.29 percentage points, respectively
AA-Omniscience v1.1 adds a public set matching its private set's difficulty
AIArtificial Analysis released a public set for AA-Omniscience v1.1 on Hugging Face, calibrated to match the difficulty of its private set. At release, public and private Net Scores correlate at 0.99 with a median difference of 0.0 points.
Source details
Artificial Analysis · x.comPublished
Original title: AA-Omniscience v1.1 has a new public set on @huggingface calibrated to match the difficulty of the private set. At release, public and pr...