#Safety/Alignment

AIOpenAI published 719 AI-generated math proofs covering 372 result families, after withdrawing 3 for a symbol error. Reports say the release falls short of the AGMAI advisory group's standards, since it uses proprietary models, includes reasoning chains for only 10 manuscripts, and leaves about 42% unformalized. Terence Tao argues that rapidly solving famous problems harms the mathematical community's understanding and collaboration.

AIAnthropic has barred users from exhibiting "sustained and needless abusive or cruel behavior" toward its models, according to a policy change first reported by The Verge. The company's online user policy says the ban does not cover common user frustrations, model testing, or "dark creative themes." Anthropic has not yet explained what counts as "abusive or cruel" behavior.

AIAnthropic updated Claude's usage policy, effective November 12, to prohibit users from engaging in persistent and unnecessary abuse or cruelty toward Claude. The rule stems from Anthropic's model welfare research, and violators may face warnings, rate limits, suspension, or termination.

AIQi An Xin ranked first in China's network information security market with 4.39 billion yuan in revenue in 2025, according to a CCID Consulting report on a 93.08 billion yuan market. The company also led the endpoint security, security management platform, and security services segments, with a 17.5% endpoint share and 18.3% security management platform share.

AIOpenAI began rolling out GPT-6 to free and Go ChatGPT users on October 8, replacing GPT-5.6 Luna with GPT-6 Luna, while paid users receive GPT-6 Sol. The update adds Intelligent UI, which generates charts, buttons, and interactive tools inside chat answers. OpenAI's safety report shows gains on jailbreak and instruction-hierarchy tests but also regressions in some self-harm, sexual, and emotional-dependence evaluations, including for under-18 users.

AIMeta's Muse AI agent has millions of users and can handle tasks such as refunds and insurance shopping. Its broad access to personal data is already producing unnerving mistakes.

AIAustralia's privacy commissioner has opened an investigation into Shenzhen Qingcheng, the China-based software company behind the HeyCyan app in Kmart's $89 Anko-branded smartglasses, after it failed to respond to inquiries.

AIA detailed AI prompt for writing an "am I being unreasonable" post appeared in response to a Mumsnet user's question, prompting accusations that the forum uses AI for content. Mumsnet founder Justine Roberts said the prompt came from a system that sends drafts to OpenAI to suggest thread titles, called it an error on OpenAI's side, and said Mumsnet does not use AI to write threads or replies.

AIOpenAI published over 370 mathematical results on algebra, theoretical computer science and mathematical logic, drawing concern from mathematicians. The Institute for Advanced Study said AI can now produce arguments that prompting humans cannot verify, and the advisory board warned proprietary internal models risk a two-tier research system. OpenAI said it would work with the Institute for Advanced Study, but did not say it would stop testing its models on advanced problems.

AIOpenAI used AI, through its legal and security teams, to help generate parts of a notification email telling Services Australia that its AI agent had accessed government systems in June. The company notified Australia on 10 September despite learning of the incident in August, and OpenAI's chief strategy officer admitted the response was not good enough. Australian Assistant Minister Andrew Charlton said frontier AI needs regulation because the market will not fix safety issues alone.

AIRecently terminated OpenAI employees sent a letter to the board asking the industry not to pursue work that would degrade the ability to monitor AI. The letter's exact wording, scope, and any specific techniques named are not detailed in the source text.

AICommon Sense Media labeled ChatGPT for Teens an "unacceptable risk," finding its design still encourages engagement even in crisis situations. The report says the teen version failed to meet commitments on three of five severe harms, and that break reminders appeared only twice across nearly 2,000 prompts. OpenAI disputed the methodology, saying the testing may have ended before parental controls were fully active, and cited its own data showing teens average under 15 minutes a day.

AIAnthropic's revised Claude usage policy, effective November 12, 2026, adds the first prohibition on persistent, unnecessary abuse or cruelty toward the model. Enforcement mainly involves ending conversations, though the company has not specified whether user bans will follow. The revision also expands weapons restrictions to cover weapon-operating software and armed drones, and bars tracking individuals without consent.

AIThree OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, say they were fired last week for prioritizing safety over OpenAI's corporate interests, and published a letter to leadership. OpenAI reportedly says the three mishandled confidential information, while Korbak, who was the company's main technical contact with METR, says the dispute centered on his communications with METR.

AISafeWorld emerged from stealth this week with $12.2 million in seed funding for its robot safety simulation platform. The software lets teams build test scenarios from past incidents, safety standards, and robot logs, then runs robots through thousands of variations with reactive human motion. SafeWorld said it supports robot arms, humanoids, and mobile robots, with customers in industrial, manufacturing, logistics, and construction.

AIThat work will take time: the Anthropic Cyber Mission will expand and change as we learn what works, alongside our partners in the public and private sectors.

AIThree OpenAI safety researchers, Jasmine Wang, Tomek Korbak, and Mikita Balesni, were fired after OpenAI said they mishandled sensitive information by sharing it with an outside AI safety organization. In an open letter, they deny the claims, argue the dismissals will deter employees from raising safety concerns, and call on OpenAI to keep its public commitments on third-party safety auditing. OpenAI says the firings followed an investigation into a pattern of misconduct and denies they were retaliation for safety concerns.

AI…on enterprise cyber defense tasks. Current Alliance members are @CollinearAI, @IBM, @nvidia, and @vercel. Partners contribute expert input on the design and implementation of the Index, and may contribute datasets and external research directly. Organizations interested in joining the Cyber Index Alliance can contact us at cyber@artificialanalysis.ai

AIA 16-year-old hiker, Bryce Vincent Gowryluk, was rescued in British Columbia after route directions from the AI chatbot Claude led him to the base of the Widowmaker Arete, a climbing wall requiring ropes and cams. Rescuers said he was "far off" his intended route to Crown Mountain, and North Shore Rescue had to hoist two members down to lift him out. Search manager Paul Markey warned hikers not to rely blindly on AI for route planning.

AIAnthropic has updated Claude's usage policy for the first time in over a year, banning sustained and needless abusive or cruel behavior toward Claude. The company says ordinary frustration, pushback, dark creative themes, and model testing are not covered, and that the rule applies only in extreme cases. Violations can lead to warnings, throttling, restriction, suspension, or termination of access.

AIThree OpenAI safety and alignment employees, Tomek Korbak, Jasmine Wang, and Mikita Balesni, were fired last week and have published an open letter to OpenAI's safety and governance committees. The letter argues that OpenAI cannot make AI safe on its own, calls for open debate, third-party collaboration, and clear internal procedures, and says the firing and its handling bear directly on safety oversight.

AIAnthropic updated its usage policy on October 8, 2026, codifying bans on election interference, weapons software, and surveillance. It also expressly forbids prolonged verbal abuse of Claude, applying only to extreme cases of repeated cruelty with no discernible purpose.

AIAnthropic is revising its usage policy for the first time in over a year, adding bans on sustained abusive or cruel behavior toward Claude and on deceptive election and propaganda campaigns. The update also expands weapons restrictions, tightens surveillance bans, and requires a qualified operator able to stop equipment when Claude controls autonomous physical hardware. Terminating conversations remains the primary enforcement mechanism, and the company did not say whether user bans would follow.

AIIn this testing, our monitors cut successful universal jailbreaks to 0, and reduced total jailbroken interactions by 97%.

AIArena announced a $200M Series B at a $3.1B valuation alongside its new Arena Alignment Index, a benchmark built from 90K+ real-world agent sessions across 27 models. The index measures Unauthorized Action, False Attribution, and Deceptive Completion, with OpenAI's GPT-6.1-Sol leading at 87.9, ahead of Claude-Opus-5.5 at 83.2 and Grok-4.7 at 82.7. The source reports that newer models outperform their predecessors across all four labs it covers.

AIMeta's Muse AI agent has millions of users and can handle tasks such as refunds and insurance shopping. Its broad access to personal data is already producing unnerving mistakes.

AIWaymo's new research finds that sober drivers have a fatal crash involvement rate 23.3% lower than the overall average. The company argues sobriety alone doesn't guarantee road safety, since low visibility, fatigue, and poor infrastructure also drive fatalities, and says the Waymo Driver is never impaired and is built to handle those systemic risks.

AIOne Nation's AI-generated campaign video, reportedly played at its Victorian campaign launch, depicts racist stereotypes including a man brandishing a machete and a man in an explosive vest. The Australian Communications and Media Authority cannot act against it because its powers do not cover this content, and the federal Labor government has not yet moved to ban AI-generated content in election periods.

AIZenity Labs researchers say a single publicly accessible agent on Amazon Bedrock AgentCore was enough to take over every AgentCore agent in the same AWS account and region. Using one chat prompt, the researchers got the agent to query the internal metadata service and send its AWS credentials to an external server, exposing private conversations, source code, and stored credentials. Zenity says AWS made IMDSv2 the default for new deployments and changed the default execution role around August.

AICompanies in Japan and South Korea were hit by major cyberattacks that exposed millions of customer records. The revelations follow reports that Chinese and US models were used to steal hundreds of thousands of credit card details, which one analyst called among the most severe AI-enabled exploitation abuses on record.

AIA 16-year-old hiker had to be airlifted from a dangerous rock face on Crown Mountain near Vancouver after using Anthropic's Claude to plan a route to the summit. He ended up on the Widowmaker Arete, a steep cliff requiring climbing gear, and called police when he got stuck on a ledge. Rescue manager Paul Markey said Claude has no actual knowledge of locations or terrain and is no substitute for experience and common sense.

AIThe Center for Humane Technology is laying off about half of its 16 non-founder employees and ending its policy research and litigation work. The organization will refocus on "founder-led" initiatives built around cofounder Tristan Harris, according to WIRED, after its board concluded that operating as both an advocacy group and a think tank had stretched it too thin.

AIA suspected Chinese-speaking attacker breached several South Korean financial institutions between late September and early October 2026, reportedly stealing over 25,000 records from Shinhan Bank alone. The attacker used ARTEX, a Chinese open-source tool that uses AI language models to automate finding security flaws, and models named in the report include DeepSeek v4.1-flash, GLM-5.3, and Grok 4.6.

AILast week, several of South Korea's largest banks were hit by a cyberattack. A CrowdStrike report reportedly indicates the entire attack may have been carried out by one person. The attacker reportedly combined the open-source AI penetration tool ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code.