Skip to content
Trending storyDeveloping

Design Arena finds AI judges favor busier pages over usable ones

2 articles1 sourcesince Nov 1Last article

Overview

AISummary of 2 articles

Design Arena reports that AI judges picked the busier page in a pairwise rewrite experiment even when told to choose the design a person would actually use.

Each pair was judged twice, once in each order, with model names hidden.

In a separate prompt test, Design Arena says a "judge by first impression" instruction lowered the judges' pick rate for the wordier website from 68% to 63%. An instruction to imagine being the person who wrote the request reached 66%. Design Arena says neither prompt brought the judges closer to human picks, which stayed at 59% to 60%.

Written by AI from the articles below · updated Nov 1, 2:47 PM ET

Check the sources:

Latest developmentNov 1, 2:31 PM ET
Design Arena judges reveal bias toward busier pages in model comparisons

Developments

2 developments

  1. Nov 1, 2:31 PM ET · 1 article
    Design Arena reports judges favored busier page designs over usable ones in a pairwise rewrite experiment
    Design Arena: Design Arena judges reveal bias toward busier pages in model comparisons
  2. Nov 1, 2:31 PM ET · 1 article
    Design Arena tests how prompt wording shifts human preference for web designs
    Design Arena: Judge-by-first-impression prompt cuts wordy-site picks from 68% to 63%

Article timeline

The articles in this story. Times are ET.

Nov 1
  1. DesignArenaOfficial
    Design Arena judges reveal bias toward busier pages in model comparisons

    AIDesign Arena says it gave judges the same instruction to pick the design a person would actually use, yet they still favored the busier page. Each pair of rewrites was judged twice, once in each order, with model names hidden.

    Image from @DesignArena's post
  2. DesignArenaOfficial
    Judge-by-first-impression prompt cuts wordy-site picks from 68% to 63%

    AIDesign Arena tested prompt wording to see whether it shifts which website people pick. A "judge by first impression" instruction lowered the pick rate for the wordier site from 68% to 63%. An "imagine you are the person who wrote this request" prompt reached 66%, and neither prompt moved the model closer to human picks, which stayed at 59% to 60%.

    Video from @DesignArena's post

Heat Trend

Current comparable heat
5
Peak comparable heat
10Nov 1, 3:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.