SWE-bench Multimodal scores barely drop when images are removed

A study by @dejavucoder reports that removing the images from SWE-bench Multimodal barely changes model scores, and that most tasks can be solved from the issue text and code alone.

The benchmark is therefore not truly multimodal, according to the post. Hamel Husain endorsed the post on X on November 3, 2026.

The finding is based on the post's own experiments as reported by Husain, and no other source is cited. The report does not name which models were tested or give specific score figures.