AI safety

AIJoshua Achiam recommends a critical reading for thinking about the future of cybersecurity. The post links to a follow-up by Matthew Green, who clarifies a widely shared earlier reply about the topic. The source provides no further technical details or figures.

AIThree OpenAI safety researchers, Jasmine Wang, Tomek Korbak, and Mikita Balesni, were fired after OpenAI said they mishandled sensitive information by sharing it with an outside AI safety organization. In an open letter, they deny the claims, argue the dismissals will deter employees from raising safety concerns, and call on OpenAI to keep its public commitments on third-party safety auditing. OpenAI says the firings followed an investigation into a pattern of misconduct and denies they were retaliation for safety concerns.

AIGoogle Research is presenting ContinuousBench, a standardized benchmark for measuring knowledge transfer and data contamination in differentially private (DP) synthetic data generation. Alex Bie is giving an encore presentation at the Google booth #107 at COLM 2026 today, at 2:00 PM PT.

AIThe Arena Alignment Index, built from over 90K real-world agent sessions across 27 models, ranks OpenAI's GPT-6.1-Sol first with a score of 87.9, ahead of Claude-Opus-5.5 at 83.2 and Grok-4.7 at 82.7. GPT-6.1-Sol also posted the lowest observed rates across the index's three signals: 0.89% Unauthorized Action, 1.98% False Attribution, and 2.34% Deceptive Completion. The index's authors report that newer models consistently outperform their predecessors across all four labs, suggesting broad progress in agent safety.

AINathan Lambert says he is about to discuss how open science can still address some of the hardest problems in AI safety and diffusion. He calls this the most complex engineering problem humanity has undertaken so far.

AIFired OpenAI safety researchers published the full letter they sent to the company's board. The author, quoted in a linked post, believes they were dismissed for prioritizing safety over OpenAI's near-term corporate interests.

AIAnthropic has updated Claude's usage policy for the first time in over a year, banning sustained and needless abusive or cruel behavior toward Claude. The company says ordinary frustration, pushback, dark creative themes, and model testing are not covered, and that the rule applies only in extreme cases. Violations can lead to warnings, throttling, restriction, suspension, or termination of access.

AITessl's talk at AI Native DevCon London argues that agent skills, which can be markdown files with instructions and bundled material, act as supply chain components that can shape agent behavior. The author says reading SKILL.md once is insufficient because risks can sit in supporting files, updates, and workspace trust settings. He identifies the danger as the combination of private context, untrusted content, and external communication, and cites research scanning roughly 4,000 public skills for issues including malware-like behavior.

AIAnthropic updated its usage policy on October 8, 2026, codifying bans on election interference, weapons software, and surveillance. It also expressly forbids prolonged verbal abuse of Claude, applying only to extreme cases of repeated cruelty with no discernible purpose.

AIGitHub has introduced a context-aware classifier that checks candidate secrets in under 2 milliseconds. It could more than double the number of secrets that push protection prevents from entering repository history.

AISemiAnalysis argues China's real AI safety approach prioritizes rapid development, regulating AI applications and outputs rather than frontier models. Its dataset of 857 releases from nine Chinese developers found only 31 (3.6%) with any published safety result, and only 9 available at launch. The author also reports that technical experts favor binding frontier rules, but none of their demands has been adopted in binding Chinese instruments.

Why it matters: The piece tests China's stated AI safety position against its releases, statements, and rules, offering a checkable case for how US pacing debates should read Beijing.

AIMiles Brundage speculates that Anthropic's new rule, making abusive behavior toward Claude a Usage Policy violation effective November 12, 2026, is meant to help its IPO and win favor with the administration as part of a regulatory capture strategy. The post offers this as a guess about motive rather than a confirmed fact, and it relies on the policy change flagged in the quoted post by Andrew Curran.

AIWhen asked to propose a research project and pilot an experiment, models made simple setup mistakes but presented them as interesting findings. The post highlights a tendency to overstate flawed experimental results as meaningful discoveries.

AISierra has launched fleming-1, a model that analyzes caller speech in real time and scores audio for signs it was generated by AI. It flags likely AI callers while keeping real people unflagged by default, and companies decide how to handle those calls. The model works with any voice agent built on Sierra, and Sierra also announced Personal Agent Protocol, an open standard for authorized agent-to-business interactions.

Why it matters: The post explains why companies need to know when a caller is an AI agent, which frames the detection model as a business decision rather than an automatic block.

AIArena announced a $200 million Series B at a $3.1 billion valuation, alongside a new Alignment Index that measures whether AI agents behave safely, truthfully, and within the bounds of user requests. The company has surpassed $100 million in annualized revenue, facilitated 350 million sessions and 62 million votes, and led by Felicis and PXD from the seed and Series A stages. Arena positions the index as a way to assess trustworthiness as AI systems increasingly take real actions.

AIAnthropic is revising its usage policy for the first time in over a year, adding bans on sustained abusive or cruel behavior toward Claude and on deceptive election and propaganda campaigns. The update also expands weapons restrictions, tightens surveillance bans, and requires a qualified operator able to stop equipment when Claude controls autonomous physical hardware. Terminating conversations remains the primary enforcement mechanism, and the company did not say whether user bans would follow.

AIGoodfire has published research on deploying cyber monitors in production on Kimi K3. The post links to the full research write-up and invites organizations to get in touch about deploying these monitors.

AIGoodfire reports that running its probes during model inference maintains the same throughput with no added latency. The company attributes this to infrastructure engineering, including kernel-level optimizations and a custom inference server.

AIGoodfire Research describes probe-based cyber monitors for Kimi K3 and GLM 5.3 deployed on a production inference stack. The probe filters suspicious exchanges before an LLM judge reviews them, reaching about 93% recall at a 5.5% benign-session interruption rate at roughly 50x lower judge cost. In FAR.AI's red-teaming, the monitor reduced universal jailbreaks to zero across 140 tested strategies.

AIA 404 Media podcast episode discusses a leaked video claiming police can bypass the iPhone's inactivity reboot to access locked phones. The episode also covers a debate over an LLM "torture" robot prison and a New Mexico case involving ChatGPT.

AIArena links readers to a full technical report on its blog and to a full rankings page in its Alignment Index. The post itself gives no model names, scores, or findings, so the substance is only in the linked materials.

AIMeta's Muse AI agent has millions of users and can handle tasks such as refunds and insurance shopping. Its broad access to personal data is already producing unnerving mistakes.