Cognition reports Devin resolves 13.86% of SWE-bench issues end to end
Original titleSWE-bench technical report
AISummary
Cognition reports that its agent Devin resolved 79 of 570 sampled SWE-bench issues, a 13.86% success rate, without being given the files to edit. The report says this exceeds the best previous unassisted baseline of 1.96% and the best assisted result of 4.80%. It also describes the adapted evaluation setup, a 45-minute runtime limit, and cases where Devin failed on multi-file edits.
AIWhy it matters
The report explains how SWE-bench was adapted for end-to-end agent evaluation, with failure cases that clarify where the 13.86% result comes from and its limits.
Source: Cognition Blog (Devin, Windsurf) · cognition.com