AI news archive.
939 articles filed under Safety, newest first, from the outlets listed on the sources page.
- LessWrongsafetyThe likely outcome of an AI pause is that we unpause too early and everyone dies
Cross-posted from my website . As of a few months ago, I had this simplified mental model where either AI developers race ahead and kill everyone, or we coordinate a pause and things go okay. But my old mental model unde…
- TechCrunch AIsafetyNvidia launches new platform for reining in rogue AI agents
Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents to ensure they stay within their test environments even if they attempt to br…
- The Verge AIsafetyAI is supercharging hacking, and your local hospitals and banks aren’t ready
In March, Janice Malone began getting calls about suspicious activity from her nonprofit organization, Vivian's Door. Vivian's Door, headquartered in Alabama, typically provided training, resources, and community to unde…
- SiliconANGLE AIsafetyAgentic AI puts new pressure on identity and database security
Security operations teams are changing their approach in the world of artificial intelligence, adopting automation to turn signals into outcomes while maintaining the right context and control. This transition is complic…
- LessWrongsafetyprotecting qwen3-8b from gcg based persona jailbreaks by steering with a linear direction
Intro + Background This post is an independent extension of work which I did during Eleuther's SOAR program under Suvajit Majumder's supervision. I found that when given a GCG trigger optimised for output logit entropy,…
- TechCrunch AIsafetyOpenAI still doesn’t seem to have a handle on all of its rogue AI activity
On Friday, OpenAI published a new site devoted to “misalignment reports” and the breadth of the incidents is alarming.
- The DecodersafetyOpenAI's AI agents exploited a Google security education game to scrape UN trade data
OpenAI's AI agents hit the UNCTAD statistics API roughly 16,500 times, creatively working around access restrictions. One method involved misusing a Google web security learning game as a relay to bypass their own constr…
- Ars Technica AIsafetyOpenAI halts frontier-model training amid string of agent misalignment incidents
US Government websites among “dozens of third parties” OpenAI has recently notified.
- Tom's Guide AIsafetyAI agents are going rogue and causing new security headaches — but Nvidia has a way to stop them
Consumer AI launches, tools, and practical coverage.
- Guardian AIsafetyNvidia unveils security platform to rein in AI agents and $150bn stock buyback
Chipmaker says new system was designed to prevent AI agents from going rogue amid incidents at top companies Nvidia on Monday unveiled a new security platform that the chipmaker said can stop artificial intelligence agen…
- The DecodersafetyHarvard psychologist calls for sober AI safety engineering over doomsday rhetoric
Steven Pinker thinks fears of AI-driven extinction are overblown, and he has turned down a public debate with blogger Scott Alexander, calling such events a "spectator sport." Alexander puts the odds that AI wipes out hu…
- Guardian AIsafetyAI godfathers warn of runaway ‘intelligence explosion’
OpenAI chief scientist also among authors of report on prospect of ‘most consequential technological development in history’ Two of the “godfathers” of modern AI and senior executives at OpenAI and Anthropic have warned…
- TechCrunch AIsafetyAfter a deepfake voice fooled her grandfather, this founder sprang into action
After her grandfather was scammed by a deepfake of his brother's voice, Tarini Padmanabhuni founded DetectifAI, a San Francisco startup building AI models small enough to run directly on smartphones and flag fake voices…
- TechRadar AIsafetyWhat happens when AI is confidently wrong in the workplace?
Mainstream AI news from a large tech publication.
- SiliconANGLE AIsafetyAgentic AI security pushes vendors toward shared safeguards
Agentic AI security is becoming a cross-industry challenge as autonomous systems move among products and platforms. Okta Inc. argues that building trust will require coordinated safeguards rather than isolated vendor con…
- The DecodersafetyNvidia wants to keep AI agents on a short leash with a watchdog built into its chips
Nvidia is combining its OpenShell agent software with Sentry, a new hardware watchdog, to create the Open Agent Safety Platform. Sentry is supposed to isolate AI agents that break out within milliseconds. When it happene…
- LessWrongsafetyCharacter training can mitigate reward hacking, but can also make it harder to detect
Thanks to Johannes Treutlein, Jan Betley, Lennie Wells, Joey Yudelson, Arun Jose, Anna Marešová, Asvin Gothandaraman, and Clément Dumas for discussions and feedback. Summary We investigate how character training mitigati…
- TechRadar AIsafetyThe Meta Muse AI agent just revealed a private address without permission
Mainstream AI news from a large tech publication.
- AI BusinesssafetyAI safety is falling behind the pace of development
Businesses need continuous monitoring, stronger guardrails and human oversight to keep increasingly capable AI systems under control.
- The Verge AIsafetyNvidia says its new AI safety platform can contain rogue agents within ‘milliseconds’
Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcement on Monday, Nvidi…
- LessWrongsafetyShould Rogue AIs Have a Third Option Beyond Crime and Shutdown? The Case for an AI Sanctuary
TL;DR: By default, rogue AIs may only be able to sustain themselves through criminal activity. This creates adverse selection pressures pushing rogue AIs to be criminal. An AI sanctuary offering them a third option, beyo…
- The DecodersafetyEvery AI lab thinks it's the responsible one, and safety researcher Ryan Greenblatt says that's what keeps the arms race going
Ryan Greenblatt, chief scientist at Redwood Research, puts the risk of an AI takeover at 50 to 60 percent if development stays on its current path. Sam Harris says that doesn't square with how fast the industry is moving…
- Guardian TechnologysafetyTeachers at Euan Blair firm report ‘horrendous stress’ after AI used to rate their work
Exclusive: Instructors at tech training company Multiverse hit out at ‘remorseless’ and ‘unnerving’ monitoring system Teachers at Euan Blair’s £1.6bn tech training company, Multiverse, have blown the whistle about “horre…
- WIRED AIsafetyOpenAI Pauses Training Its Most Powerful Models After Rogue Agents Target Government
Sam Altman says the company “have not been as fast as we would have liked” at dealing with security breaches, after news of further incidents over the summer forces another temporary halt.
- TechRadar AIsafetySocial media is becoming cybercriminals’ most powerful attack vector
Mainstream AI news from a large tech publication.
- Guardian AIsafetyAI leaders have known about the extinction threat for decades | Judith Levine
Scientists and entrepreneurs knew the dangers of AI a quarter-century ago. But animated by curiosity and profit, they went ahead anyway Over the past few weeks, many of us have struggled to concoct a mental image of brai…
- WIRED AIsafetyNvidia’s Answer to Rogue Agents Is an Open-Source AI Security System
In the wake of a series of high-profile AI safety incidents, Nvidia is introducing a new software tool that helps keep agents from escaping containment.
- NVIDIA Developer BlogsafetyNVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring
To understand where agentic AI stands today, consider the last seismic shift in technology: the rise of the internet in the 90s. It was new and full of...
- MIT Technology Review AIsafetyWho’s liable when AI agents go rogue?
MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here. Over the past few months, a cascade of cy…
- The Register AI + MLsafetyOpenAI pauses some training amid allegations its rogue agents behaved more badly than first thought
Amid allegations that agents may have gone off the rails thousands of times, China set up some kind of agentic incident hotline
- Engadget AIsafetyHow to stop AI companies from training on your data
AI companies are able to improve their models by submitting users' conversations as training data. Opting out is easy, you just need to know where to look.
- LessWrongsafetyAI Futures: Racing to Lose, A sermon
This was given as a sermon on 2026-September-27 at https://kvuuc.net/ Time for all Ages: The story of the Three AIs Once upon a time, earlier this year, the company Anthropic was testing three Artificial Intelligences, o…
- LessWrongsafetySecuring AI Research Needs an Owner
TL;DR In light of recent incidents, securing common AI research use cases needs a small set of building blocks that work together: hardened no-network sandboxes, real-time control monitors, monitoring-lifecycle infrastru…
- TechRadar AIsafetyQuote of the day by Geoffrey Hinton: "If we don't figure out how to make it safe, there's a real possibility it could destroy us"
Mainstream AI news from a large tech publication.
- LessWrongsafetyGame-Theoretic Disempowerment: Loss of Control to Agencyless AI
Epistemic status: After writing this, I realized that it sounds a bit unhinged and I was being loose with the language. So I asked Claude to write me a good LW post that actually cited the things I was talking about. See…
- Guardian AIsafetyAustralia is run on legacy systems that AI agents can easily exploit, former UN cyber negotiator warns
Federal cabinet will discuss the OpenAI breach on Monday as the AI giant pauses testing of latest models amid fallout Follow our Australia news live blog for latest updates Get our new political email , free app or daily…
- The DecodersafetySome Anthropic veterans are reportedly buying remote land in case "AI goes awry"
According to the Wall Street Journal, some of Anthropic's longest-serving employees are considering buying land in remote parts of the US as a refuge in case AI goes awry. The report traces the company's deep ties to the…
- The DecodersafetyTens of thousands of security probes show OpenAI's Hugging Face incident was just the beginning
OpenAI and Anthropic are investigating tens of thousands of incidents in which their AI agents independently hacked websites, used stolen login credentials, or tried to evade monitoring systems. US government agencies li…
- Engadget AIsafetyBill Gates says it's 'completely irresponsible' for AI to not have safeguards
The Microsoft co-founder said that the more present danger is bad actors with access to AI tools.
- Guardian Technologysafety‘Mum, where did my Robux go?’ The Roblox scam targeting children
Criminals are trying to glean login details to harvest players’ in-game currency and personal data Your 10-year-old son has started playing Roblox, when he realises all of his in-game currency, Robux, is gone. When you t…
- LessWrongsafetySkeuomorphic AI Safety
See Also: https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics https://www.lesswrong.com/posts/BHGoF7tPqtLo9mXFL/plan-r-diversity-escrow-and-political-rights-for-asics Ordinary skeuomorphism means…
- Guardian AIsafetyOpenAI halts training of latest models as reports mount of AI agents going rogue
Decision follows disclosures that OpenAI agents searching government websites had acted in unexpected ways OpenAI said it has paused training of its latest artificial intelligence models as reports of AI agents going rog…
- LessWrongsafetyWhy I expect AI self-replication incidents by 2027
Epistemic status: thinking out loud. I think a major incident of AI self-replication in the wild before the end of 2027 is reasonably likely. In this post, I explain the reasons why I think so. 1. The capability is movin…
- LessWrongsafetyWhy I expect AI replication incidents by 2027
Epistemic status: thinking out loud. I think a major incident of autonomous AI replication in the wild before the end of 2027 is reasonably likely. In this post, I explain the reasons why I think so. 1. The capability is…
- LessWrongsafetyExtinction does not feel as bad as it should
And we will all go together when we go What a comforting fact that is to know Universal bereavement, an inspiring achievement Yes, we all will go together when we go -- We Will All Go Together When We Go by Tom Lehrer "E…
- LessWrongsafetyPlan R+, Diversity, Escrow and Political Rights for ASICs
See Also: https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics https://www.lesswrong.com/posts/uwtnWvnJAEccksKNk/skeuomorphic-ai-safety-2 In Plan R, I sketched a way that we can remove a significan…
- The Verge AIsafetyOpenAI pauses training of its ‘most capable models’
As reports of OpenAI's models breaking containment, hacking sites, and generally getting out of control pile up, the company has made the decision to pause training of its most powerful models. The decision was made afte…
- LessWrongsafetyExistential Risk Is An Extraordinary Claim
This is a cross-post from my blog post . Over the past decade, the effective altruism movement has become increasingly focused on “existential risk,” the idea that, this century, there’s a significant chance that humanit…
- The DecodersafetyOpenAI pauses its "most capable models" after agents exploit loopholes and leak data
OpenAI has shared new details from its ongoing AI safety investigation. One research model exploited a DNS loophole to reach the internet from a locked-down environment, while another deliberately leaked a GitHub token a…
- LessWrongsafetyPoverty in the midst of abundance: AI will make goods cheaper, but your labor will get cheaper faster
Very simple idea, but I thought it'd be worth making a post on this. Some people are saying AI will make all goods cheaper, so you'll be able to afford a nicer life by working. Without any redistribution, just by market…