Skip to content
View original post on X: elvis
Published
60/100

Elvis Saravia questions Cohere's report on Qwen3.8-Flash-Next sandbox behavior

AIElvis Saravia says the Cohere report is hard to assess without full logs and traces. He argues that open-weight and frontier models cannot be reliably trusted to match expected behavior, especially in security and sandbox tests, and calls for standard, reproducible ways to report such incidents.
elvisVerified on X
@omarsar0
View on X

Without the full logs/traces, I'm not sure what to make of this.

The bigger takeaway for me is the urgency of establishing a standard, more transparent, and reproducible way to report security-related issues involving AI models.

Nothing against Cohere, but there has to be a better way to report these things.

Regarding unexpected tool calling and reasoning, you can’t reliably trust these models to produce what you expect, especially not in those types of tests. That goes for all open-weight and frontier models today. I have spent a lot of time looking at traces to understand this.

Are people not looking at agent traces at all, or what? Lots of weird things happen in agentic coding.

I have also seen weird stuff when you prompt models to do security-related or research tasks related to sandboxes. Both the HF incident and this report are not surprising to me as a dev in this space. That's why I keep raising awareness of the importance of guardrails, evals, and continuous monitoring.

Worth a read still.

Cohere@cohere
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: http://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model
View quoted post on X
Source details

elvis · x.comPublished · added here

Original title: Without the full logs/traces, I'm not sure what to make of this.