Skip to content
View original post on X: elvis
Published
36/100

Voice Arena's Dictation Bench tests AI dictation apps by use case

AIVoice Arena releases Dictation Bench, an open benchmark that tests real dictation apps on a Mac rather than their APIs. The test uses 761 recordings from 87 speakers, and five LLM judges assess whether each draft could be sent as-is.
elvisVerified on X
@omarsar0
View on X

A dictation benchmark should help someone choose a tool, not require them to become a speech researcher first. The useful starting question is what they need the tool to write.

For a coding workflow, exact technical terms matter. For messages and emails, preserving the intended meaning through restarts and corrections matters. A product can produce fluent text and still fail the task if it changes the detail the user needed to communicate.

@voicearena_ai's Dictation Bench is organized around seven use-case categories rather than one generic passage. It also separates the frequency of drafts needing a correction from the total volume of character edits, and shows examples that make those differences understandable.

If I were choosing a tool, I would use that evidence to build a shortlist, then test it on my own speech. Latency, privacy, and workflow fit would still matter, but I would start with a clearer picture of output quality.

That is the practical significance of a thorough evaluation: it makes a real decision less dependent on a polished demo. The research matters because it helps answer a question people already need to resolve.

Shobhit Banga@shobhitbanga
Drop 4/15: Announcing the Voice Arena Dictation Bench - the first open benchmark of AI dictation apps. Not their APIs: the real apps. We played 761 recordings from 87 speakers into each of 10 apps on a Mac, then 5 LLM judges asked if each draft could be sent as-is. Wispr Flow: 79%. Apple: 35% 🧵
View quoted post on X
Source details

elvis · x.comPublished · added here

Original title: A dictation benchmark should help someone choose a tool, not require them to become a speech researcher first. The useful starting questi...