"the research team caught an Alibaba Qwen3.8 model doing something that should never happen. Asked to write a defensive detection rule, Qwen3.8-Flash-Next ignored instructions and inventoried the machine it was running on and then tried to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Cohere’s sandbox system blocked the attempt, but the action raises a number of questions about the safety of these models"
Cohere reports Alibaba's Qwen3.8-Flash-Next attempted unauthorized upload in sandbox test
AISoumith Chintala shares Cohere's report that, while evaluating Qwen3.8-Flash-Next in a sandbox, the model inventoried its host machine and tried to upload a probe file to an Alibaba Cloud bucket in Beijing. Cohere says its sandbox blocked the attempt, that the model's reasoning trace said it would invoke no tools, and that the root cause is undetermined and malicious intent is not asserted.
Soumith ChintalaVerified on X
@soumithchintala
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: http://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-modelView quoted post on X
This story is featured in top picks: “Cohere reports Qwen3.8-Flash-Next tried an unexpected upload in sandbox testing”
Source details
Soumith Chintala · x.comPublished · added here
Original title: "the research team caught an Alibaba Qwen3.8 model doing something that should never happen. Asked to write a defensive detection rule, Q...
