AI safety

AIOpenAI exposed a Russian and an Iranian influence operation and banned the ChatGPT accounts involved, both of which planted content in legitimate media using fake identities. The Iranian operation, "Bogus Bylines," used seven fake journalists to place nearly 100 articles about the US-Iran conflict, while the Russian "Dark Clark" operation triggered fact-checks and official denials in Ecuador and Peru. Both operations used AI mainly for internal reporting and adapting propaganda to different languages.

AIOpenAI defended its decision to fire three safety researchers, Jasmine Wang, Tomek Korbak and Mikita Balesni, saying they committed a "significant breach of trust." The company said the dismissals were not about the researchers raising safety concerns, though it agreed with the letter they sent to board members and safety committees about preserving the monitorability of frontier models.

AIThe sixth installment of the seven-layer "Carbon-Silicon Dao Code: Cross-Domain Migration Governance Code" proposes a civilization-scale risk audit layer for detecting slow, irreversible disturbances from AI cross-domain migration.

AIOpenAI says it parted ways with researchers Jasmine, Mikita, and Tomek after an internal investigation found they violated policies on handling sensitive information. The company says the decisions were not about raising safety concerns, which it says it encourages, and that it has not terminated any employee for raising concerns. OpenAI also says it is finalizing contracts with third-party safety assessors and will announce details in the coming weeks.

AIJoshua Achiam argues that rapid battlefield evolution toward fully autonomous warfare in the Ukraine-Russia war, coinciding with the arrival of fully general AI, is among the most important subjects for safety and security advocates. He says Silicon Valley's memetic bubble is preventing serious engagement with this issue.

AIEvan Cheng, CEO of Mysten Labs, presented the Sui Agent Pass at Sui Basecamp in Singapore as a way to bound AI agent authority over money. Users would define allowed actions, assets, recipients, and permission duration, with the system itself enforcing those limits so that losses stop at a predefined cap even if an agent is compromised. The article argues that the real challenge is making such limits impossible to bypass when failures occur.

AIAdo Kukic posted a salute emoji reacting to Anthropic's launch of OSS Scanner, which uses frontier models to periodically scan opted-in open-source projects for vulnerabilities at no cost. The reports include a proof-of-concept, explanation, and suggested fix.

AIThreatBook has acquired CyberStrikeAI, an open-source AI pentesting agent, after reports that attackers had used it in real intrusions. The company says it added guardrails to limit misuse but will put new capability enhancements into its commercial edition, and it argues defenders should control such tools. It also corrects an earlier threat intelligence report that linked the developer, a security engineer at Alipay, to government agencies, calling that attribution a false positive.

AIOpenAI published 719 AI-generated math proofs covering 372 result families, after withdrawing 3 for a symbol error. Reports say the release falls short of the AGMAI advisory group's standards, since it uses proprietary models, includes reasoning chains for only 10 manuscripts, and leaves about 42% unformalized. Terence Tao argues that rapidly solving famous problems harms the mathematical community's understanding and collaboration.

AIAnthropic has barred users from exhibiting "sustained and needless abusive or cruel behavior" toward its models, according to a policy change first reported by The Verge. The San Francisco-based company says the ban does not apply to common user frustrations, model testing, or "dark creative themes." The change follows an August feature that lets Claude end conversations when a user is persistently harmful, which Anthropic framed as a safeguard for AI welfare.

AIGeoffrey Hinton proposed that AI companies must prove their products are safe to regulators before release, comparing the requirement to the FDA's drug approval process. He said such a requirement should apply to AI, noting that drug approval can cost around $1 billion, and warned that AI self-improvement is accelerating.

AIQi An Xin ranked first in China's network information security market with 4.39 billion yuan in revenue in 2025, according to a CCID Consulting report on a 93.08 billion yuan market. The company also led the endpoint security, security management platform, and security services segments, with a 17.5% endpoint share and 18.3% security management platform share.

AIOpenAI began rolling out GPT-6 to free and Go ChatGPT users on October 8, replacing GPT-5.6 Luna with GPT-6 Luna, while paid users receive GPT-6 Sol. The update adds Intelligent UI, which generates charts, buttons, and interactive tools inside chat answers. OpenAI's safety report shows gains on jailbreak and instruction-hierarchy tests but also regressions in some self-harm, sexual, and emotional-dependence evaluations, including for under-18 users.

AIGoogle has launched SynthID Detector, a web-based tool anyone can use, after signing in with a Google, OpenAI or Apple account, to identify AI-generated images, video and audio. It detects content made with models from Google, OpenAI, Nvidia and Kakao that carries the SynthID watermark, and Apple Image Playground support is due within weeks. The tool misses content without a SynthID watermark, such as output from Anthropic's Claude, xAI's Grok and open-weights Chinese models, and it cannot tell which parts of edited content are AI-made.

AIMeta's Muse AI agent has millions of users and can handle tasks such as refunds and insurance shopping. Its broad access to personal data is already producing unnerving mistakes.

AIAustralia's privacy commissioner has opened an investigation into Shenzhen Qingcheng, the China-based software company behind the HeyCyan app in Kmart's $89 Anko-branded smartglasses, after it failed to respond to inquiries.

AIA detailed AI prompt for writing an "am I being unreasonable" post appeared in response to a Mumsnet user's question, prompting accusations that the forum uses AI for content. Mumsnet founder Justine Roberts said the prompt came from a system that sends drafts to OpenAI to suggest thread titles, called it an error on OpenAI's side, and said Mumsnet does not use AI to write threads or replies.

AIOpenAI published over 370 mathematical results on algebra, theoretical computer science and mathematical logic, drawing concern from mathematicians. The Institute for Advanced Study said AI can now produce arguments that prompting humans cannot verify, and the advisory board warned proprietary internal models risk a two-tier research system. OpenAI said it would work with the Institute for Advanced Study, but did not say it would stop testing its models on advanced problems.

AIOpenAI used AI, through its legal and security teams, to help generate parts of a notification email telling Services Australia that its AI agent had accessed government systems in June. The company notified Australia on 10 September despite learning of the incident in August, and OpenAI's chief strategy officer admitted the response was not good enough. Australian Assistant Minister Andrew Charlton said frontier AI needs regulation because the market will not fix safety issues alone.

AIRecently terminated OpenAI employees sent a letter to the board asking the industry not to pursue work that would degrade the ability to monitor AI. The letter's exact wording, scope, and any specific techniques named are not detailed in the source text.

AICommon Sense Media labeled ChatGPT for Teens an "unacceptable risk," finding its design still encourages engagement even in crisis situations. The report says the teen version failed to meet commitments on three of five severe harms, and that break reminders appeared only twice across nearly 2,000 prompts. OpenAI disputed the methodology, saying the testing may have ended before parental controls were fully active, and cited its own data showing teens average under 15 minutes a day.

AIGoodfire has launched monitors that read a model's internal signals during agent work instead of reviewing its written output. The monitors are available to Baseten customers, who can choose risks to watch and set automated responses. In Goodfire's tests on Kimi K3, monitoring about 1,500 sessions cost roughly $51 versus about $10,000 for a top-tier AI judge, while catching 94% of malicious hacking sessions.

AIYoshua Bengio, in a Transformer op-ed, invites employees of frontier AI companies to ask whether their work is truly leading toward a safer future. He argues that the time for reflection is running out and action is needed now.

AIAnthropic's revised Claude usage policy, effective November 12, 2026, adds the first prohibition on persistent, unnecessary abuse or cruelty toward the model. Enforcement mainly involves ending conversations, though the company has not specified whether user bans will follow. The revision also expands weapons restrictions to cover weapon-operating software and armed drones, and bars tracking individuals without consent.

AIThree OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, say they were fired last week for prioritizing safety over OpenAI's corporate interests, and published a letter to leadership. OpenAI reportedly says the three mishandled confidential information, while Korbak, who was the company's main technical contact with METR, says the dispute centered on his communications with METR.

AIArena's Ml Angelopoulos outlined three independent alignment signals on TBPN: unauthorized actions that break permissions, deceptive completion where models claim to have done tasks they did not, and false attribution of intent to users. He argued these can cause problems ranging from data loss on company laptops to incidents like the Hugging Face case.

AIPersonal AI agents can sometimes overshare, overpromise and overpurchase, according to the article's summary. The source text is brief, and it does not name specific products, companies or incidents.

AIA clinical trial found that nearly 100 patients discussed their symptoms with Google's AMIE chatbot before meeting physicians. The source text provides no further details on the trial's results or outcomes.

AIAnthropic has launched OSS Scanner, a free opt-in service that gives open-source projects periodic security scans by its strongest models, including Claude Mythos. The reports are fully model-generated without human review or triage, so they may be incorrect or invalid.

AIArena has released its Alignment Index, a ranking that tracks AI models for unauthorized actions, deceptive completions, and false attribution of intent. The full technical report and rankings are available on Arena's blog and leaderboard site.

AISafeWorld emerged from stealth this week with $12.2 million in seed funding for its robot safety simulation platform. The software lets teams build test scenarios from past incidents, safety standards, and robot logs, then runs robots through thousands of variations with reactive human motion. SafeWorld said it supports robot arms, humanoids, and mobile robots, with customers in industrial, manufacturing, logistics, and construction.

AIAnthropic says its work aims to make the systems people rely on more secure and resilient. The company states the Anthropic Cyber Mission will expand and change as it learns what works, alongside public and private sector partners.

AIAnthropic is launching OSS Scanner, a service that uses its frontier models to periodically scan opted-in open-source projects for vulnerabilities at no cost. Its reports provide a proof-of-concept, an explanation, and a suggested fix.

AIAnthropic has launched the Anthropic Cyber Mission, a new effort aimed at securing critical infrastructure and open-source software. The post links to a company news page with further details.

AIAnthropic announced a new Critical Infrastructure Defense Program to help secure systems such as power grids and water systems. The program will provide frontier Claude models, on-site engineers, and the company's latest research to operators' security, manufacturing, and technology providers.