Skip to content
Trending storyDeveloping

Dictation Bench: Voice Arena's open test of Mac dictation apps

2 articles2 sourcessince Nov 11Last article

Overview

AISummary of 2 articles

Voice Arena releases Dictation Bench, an open benchmark that runs the same 761 recordings from 87 speakers through ten Mac dictation apps and publishes its policies, configurations, judge instructions and results.

Five LLM judges assess whether each draft could be sent as-is. Rohan Paul says dictation companies should check any "best" accuracy claim against the benchmark and report against it routinely. Paul notes the current scope is US English on macOS only.

Elvis Saravia says the benchmark tests real dictation apps rather than their APIs, and reports that Wispr Flow scored 79% and Apple scored 35%. He argues users should choose tools by the output quality their own tasks need. The source reports these scores, but the story does not report independent verification of them.

Written by AI from the articles below · updated Nov 11, 11:56 AM ET

Check the sources:

Developments

2 developments

  1. Nov 11, 11:35 AM ET · 1 article
    Voice Arena's Dictation Bench tests AI dictation apps by use case
    Elvis Saravia: Voice Arena's Dictation Bench tests AI dictation apps by use case
  2. Nov 11, 10:09 AM ET · 1 article
    Rohan Paul calls for AI dictation apps to be judged on Dictation Bench
    Rohan Paul: Rohan Paul calls for AI dictation apps to be judged on Dictation Bench

Article timeline

The articles in this story. Times are ET.

Nov 11
  1. elvisX
    Voice Arena's Dictation Bench tests AI dictation apps by use case

    AIVoice Arena releases Dictation Bench, an open benchmark that tests real dictation apps on a Mac rather than their APIs. The test uses 761 recordings from 87 speakers, and five LLM judges assess whether each draft could be sent as-is. The post says Wispr Flow scored 79% and Apple scored 35%, and argues that users should pick tools based on the output quality their specific tasks need.

  2. Rohan PaulX
    Rohan Paul calls for AI dictation apps to be judged on Dictation Bench

    AIRohan Paul says a dictation company's "best" claim should be checked against Dictation Bench, a shared test from Voice Arena. The benchmark plays the same 761 recordings from 87 speakers into ten Mac dictation apps and publishes its policies, configurations, judge instructions and results. Paul says the current US English, macOS-only scope should be kept in mind, and he wants companies to report against the benchmark routinely.

Heat Trend

Current comparable heat
12
Peak comparable heat
19Nov 11, 12:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.