More model launches should publish their benchmark harness.
AirCaps just launched A5S Async v1, a speech-to-text model that tracks who said each word. They also published the code, dataset recipes, raw API responses, and scores behind their results.
It has the lowest error rate on all 5 public meeting benchmarks they tested.
Introducing A5S Async V1: the world's most accurate speech recognition model for speaker-attributed transcription. #1 on VoiceArena, ahead of ElevenLabs, AssemblyAI, Deepgram, NVIDIA, Meta & more. The next-best model makes 41% more errors across 5 public benchmarks. And yes, every lab says they're #1. So we open-sourced our benchmark. Try it now👇
elvis · x.comPublished · added here
Original title: More model launches should publish their benchmark harness.
