#Coding

AICognition has released Devin 2.1, which reports its confidence in completing tasks using green, yellow, and red ratings. The company says green scores led to twice the likelihood of a merged PR compared with red, and Devin now also answers codebase questions and scores Linear and Jira issues.

Why it matters: The post explains how Devin now shows confidence scores and asks clarifying questions, which changes how teams can decide which tasks to hand over.

AIStanford and Cognition AI researchers introduced Kevin-32B, a 32B-parameter model trained with multi-turn reinforcement learning to write CUDA kernels. On KernelBench, it solves 89% of tasks at best@16 and achieves 65% average correctness over eight refinement steps, versus 53% for o4-mini and 51% for o3. Its best@16 speedup is 1.41x, and multi-turn training outperforms single-turn training as refinement steps increase.

AICognition has launched DeepWiki, a free public version of its Devin Wiki and Devin Search tools that helps developers understand codebases. Users can view docs for any repo by replacing github.com with deepwiki.com in the URL, and more than 50,000 top public GitHub repositories are already indexed. Private repositories require a Devin account.

AICognition has released Devin 2.0, a new agent-native IDE experience with a flexible plan starting at $20. The update lets users run multiple parallel Devins, each with its own cloud-based IDE, and adds Interactive Planning, Devin Search, and Devin Wiki.

Why it matters: The release adds planning, codebase search, and auto-generated wikis to Devin, showing how an agent can prepare work before executing it.

AICognition's February 2025 Devin update adds parallel batch edits, beta GitLab support, and Sonnet 3.7, which Cognition says is the best model it has tested for debugging, codebase search, and agentic planning. Devin is about 2x faster than in October 2024, taking about 7.8 minutes on average to complete junior developer tasks in Cognition's internal evaluations. Other changes include copy-paste in Devin's browser and proactive feedback on suboptimal prompts.

AILinktree has used Devin, Cognition's AI software engineer, to merge roughly 100 pull requests in a month, mostly fixing customer-reported bugs and implementing small features. The engineering team also used Devin to add support for new social media platforms, launching five Devins, one per repo and PR, and later used the Devin API with a Playbook script to spawn multiple Devins for multi-repo features. The team says Devin works best on tasks an engineer could finish in a couple of hours.

AICognition's Devin can be triggered through its External API by GitHub Actions to automatically review pull requests, typically within five to ten minutes. The setup involves adding a workflow file, storing a DEVIN_API_KEY secret, and customizing the review prompt to match team conventions. Cognition recommends treating Devin as an extra reviewer rather than a replacement for human oversight, since it does not catch every bug.

AICognition's January 2025 Devin update improves its ability to find relevant files and reuse existing code in repositories, with changes rolling out to all users. It also adds enterprise accounts for centralized management of multiple organizations, audio message support in Slack, and pay-as-you-go billing after monthly ACU capacity is used, starting January 9.

AICrossmint said Devin became its top contributor to the open-source GOAT SDK during an initial trial, merging 8 pull requests versus 4 for the next contributor. Examples included a DEXScreener plugin built from a documentation URL and a harder Sui blockchain integration that needed three rounds of feedback and about an hour of human involvement. The company said its results depended on proper training, clear task context, and planned validation, not on treating Devin as superhuman.

AICognition has made Devin generally available to all engineering teams, with subscriptions starting at $500 per month. Over the past two weeks, Devin was made about 10% faster and about 10% more cost-efficient, especially for tasks requiring many code edits. The update also adds fixes for stuck or hanging sessions, more options to customize filters and Slack notifications, and larger machine settings for disk, RAM, and CPU.

AICognition is launching the Devin Open Source Initiative, giving selected open source maintainers 500 free ACUs on a Devin Teams plan as part of Devin's general availability launch. The post shows Devin contributing pull requests to projects including Anthropic's MCP Inspector, Dagger, and nanoGPT, with maintainers still reviewing the results. Devin's GitHub integration forwards PR comments and CI checks to help refine changes, though the company warns a human should still verify final quality.

AICognition is making Devin generally available to engineering teams starting at $500 a month, with no seat limits and access to its Slack integration, IDE extension, and API. The post recommends starting with small frontend bugs, first-draft PRs for backlog tasks, and targeted refactors, and shares open-source PR sessions where Devin resolved issues for projects including Anthropic MCP, Zod, and nanoGPT.

Why it matters: The post shows concrete open-source PR examples and the tasks where Devin works best, helping teams judge where an autonomous coding agent fits their workflow.

AICognition's December 2024 Devin update lets users tag Devin in Slack threads and ask it to create PRs, with Devin automatically responding to PR comments and lint failures. The update adds Repo Knowledge that Devin generates by scanning repositories, an Agency setting that makes Devin propose plans before executing complex tasks, and a REST API for structured input and output. Devin also gained faster session startup and enterprise options such as Okta single sign-on.

AICognition tested OpenAI's o1-mini and o1-preview in a simplified Devin-Base agent, comparing them with GPT-4o on its internal cognition-golden benchmark. The chart reports Devin-Base scores of 25.9% with GPT-4o, 34.6% with o1-mini, and 51.8% with o1-preview, versus 74.2% for the production Devin. The post also describes the benchmark's realistic environments, simulated users, and agent-based evaluation.

Why it matters: The post explains how Cognition evaluates coding agents with autonomous, environment-based tests, which shows how base-model swaps are measured in practice.

AICognition's Devin gained a September 2024 update with MultiDevin, which lets a manager Devin delegate work to up to 10 worker Devins, currently available on the Enterprise plan. Devin now automatically responds to comments on its pull requests, suggests Knowledge additions, and can restore earlier checkpoints, and the company reports up to an 80% reduction in time for common tasks.

AIDevin's June 2024 update lets users directly operate its VSCode editor, terminal, and browser, and adds Playbooks for recurring engineering tasks. It also adds Knowledge sharing, Machine Snapshots that persist dev environments, event-driven triggers from Slack, GitHub, and Linear, a Secrets manager, and tools for verifying Devin's work. Access remains limited to a waitlist with weekly invite releases.

AICognition reports that its agent Devin resolved 79 of 570 sampled SWE-bench issues, a 13.86% success rate, without being given the files to edit. The report says this exceeds the best previous unassisted baseline of 1.96% and the best assisted result of 4.80%. It also describes the adapted evaluation setup, a 45-minute runtime limit, and cases where Devin failed on multi-file edits.

Why it matters: The report explains how SWE-bench was adapted for end-to-end agent evaluation, with failure cases that clarify where the 13.86% result comes from and its limits.

AICognition introduces Devin as an AI software engineer that can plan and execute complex engineering tasks with a shell, code editor, and browser. On SWE-bench, Devin resolved 13.86% of issues end-to-end, versus a previous state-of-the-art of 1.96%, on a random 25% subset of the dataset. Devin is in early access, with access available through a waitlist.

Why it matters: The post pairs Devin's end-to-end task demos with SWE-bench results against prior models, letting readers weigh the claimed capability against the evaluation setup.