Skip to content
View original post on X: BAAIOfficial· 12/100AI score12/100

Study finds visual quality, drift, and response speed do not track interaction reliability

AISummary

BAAI reports that IAMFlow scores highest on visual quality at 2.840 but shows the largest quality drift at 0.425. Causal-Forcing has lower absolute quality yet the lowest visual drift at 0.189. SWIFT responds in 10.3 seconds but succeeds in only 24.6% of updates, meaning fast responses can mask ignored user instructions.

Post on XView on X
BAAIVerified on X
@BAAIBeijing

Part of a thread · earlier post

Crucial technical insights:

• High Quality ≠ Long-Term Stability: IAMFlow hits top visual quality (2.840) but suffers large quality drift (0.425). Causal-Forcing has lower absolute quality but achieves the lowest visual drift (0.189).

• Fast Response ≠ Reliable Interaction: SWIFT clocks a swift 10.3s response time, but only achieves a 24.6% update success rate. Low latency without high achievement masks ignored user instructions.

Source: BAAI · x.comPublished · added here

Topics

AI videoPapers & researchEvals & benchmarks