MKA tests models’ visual understanding of documents including engineering schematics, architectural drawings, and clinical scans and how they leverage these to produce slide decks, spreadsheets, reports, and marked-up images.
In the below example task from the 10-task public dataset, models audit a building's plans. This includes checking sections and elevations for door and window mismatches, and reporting the findings as a slide deck.
Claude Fable 5.2 (max) takes 224 turns to complete the deliverable, including viewing images 203 times, costing a total of ~$68 for the single task.