home.social

#ai-safety — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #ai-safety, aggregated by home.social.

fetched live
  1. "Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

    In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

    My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

    theatlantic.com/ideas/2026/09/

    #AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

  2. "Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

    In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

    My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

    theatlantic.com/ideas/2026/09/

    #AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

  3. "Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

    In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

    My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

    theatlantic.com/ideas/2026/09/

    #AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

  4. "Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

    In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

    My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

    theatlantic.com/ideas/2026/09/

    #AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

  5. "Follow the connections out from that room and you pass through some genuinely strange territory. A Harry Potter fanfic used as a recruiting pipeline. Psychological workshops that former participants describe as coercive. A program for breeding smarter children, promoted by an AI institute. A sex worker from Idaho who became one of the subculture’s most-read writers and now runs an AI-doom propaganda residency with Grimes, Elon Musk’s ex-girlfriend. A splinter sect that fled to tugboats and whose members have been linked to six deaths. And a blogger who wanted to abolish democracy, found the ear of several billionaire patrons, and watched his ideas come to fruition in the vice president’s office.

    I am aware, that if you are not lurking on the internet as much as I do, of how utterly insane that paragraph sounds. However, every bit of it is heavily documented by numerous sources. The people in this story are not a secret society coordinating over dinner. They are a social scene: overlapping friendship groups, funding relationships, and sexual and romantic relationships, all of it threaded through the institutions asking you to trust them about AI regulation.

    One of many unfortunate shortcomings of mainstream media is that credibly reporting on these things is so absurd that you can’t expect readers to take it seriously. However, I’ll endeavor to provide a thorough account of how our current slide into neo-fascism, theocratic dystopia, and the forecasted end of the world is deeply rooted in the history of this subculture."

    iankduncan.com/personal/2026-0

    #AI #AIDoomsters #AIDoom #AIApocalypse #AISafety #Ideology #SiliconValley

  6. "Follow the connections out from that room and you pass through some genuinely strange territory. A Harry Potter fanfic used as a recruiting pipeline. Psychological workshops that former participants describe as coercive. A program for breeding smarter children, promoted by an AI institute. A sex worker from Idaho who became one of the subculture’s most-read writers and now runs an AI-doom propaganda residency with Grimes, Elon Musk’s ex-girlfriend. A splinter sect that fled to tugboats and whose members have been linked to six deaths. And a blogger who wanted to abolish democracy, found the ear of several billionaire patrons, and watched his ideas come to fruition in the vice president’s office.

    I am aware, that if you are not lurking on the internet as much as I do, of how utterly insane that paragraph sounds. However, every bit of it is heavily documented by numerous sources. The people in this story are not a secret society coordinating over dinner. They are a social scene: overlapping friendship groups, funding relationships, and sexual and romantic relationships, all of it threaded through the institutions asking you to trust them about AI regulation.

    One of many unfortunate shortcomings of mainstream media is that credibly reporting on these things is so absurd that you can’t expect readers to take it seriously. However, I’ll endeavor to provide a thorough account of how our current slide into neo-fascism, theocratic dystopia, and the forecasted end of the world is deeply rooted in the history of this subculture."

    iankduncan.com/personal/2026-0

    #AI #AIDoomsters #AIDoom #AIApocalypse #AISafety #Ideology #SiliconValley

  7. "Follow the connections out from that room and you pass through some genuinely strange territory. A Harry Potter fanfic used as a recruiting pipeline. Psychological workshops that former participants describe as coercive. A program for breeding smarter children, promoted by an AI institute. A sex worker from Idaho who became one of the subculture’s most-read writers and now runs an AI-doom propaganda residency with Grimes, Elon Musk’s ex-girlfriend. A splinter sect that fled to tugboats and whose members have been linked to six deaths. And a blogger who wanted to abolish democracy, found the ear of several billionaire patrons, and watched his ideas come to fruition in the vice president’s office.

    I am aware, that if you are not lurking on the internet as much as I do, of how utterly insane that paragraph sounds. However, every bit of it is heavily documented by numerous sources. The people in this story are not a secret society coordinating over dinner. They are a social scene: overlapping friendship groups, funding relationships, and sexual and romantic relationships, all of it threaded through the institutions asking you to trust them about AI regulation.

    One of many unfortunate shortcomings of mainstream media is that credibly reporting on these things is so absurd that you can’t expect readers to take it seriously. However, I’ll endeavor to provide a thorough account of how our current slide into neo-fascism, theocratic dystopia, and the forecasted end of the world is deeply rooted in the history of this subculture."

    iankduncan.com/personal/2026-0

    #AI #AIDoomsters #AIDoom #AIApocalypse #AISafety #Ideology #SiliconValley

  8. "Follow the connections out from that room and you pass through some genuinely strange territory. A Harry Potter fanfic used as a recruiting pipeline. Psychological workshops that former participants describe as coercive. A program for breeding smarter children, promoted by an AI institute. A sex worker from Idaho who became one of the subculture’s most-read writers and now runs an AI-doom propaganda residency with Grimes, Elon Musk’s ex-girlfriend. A splinter sect that fled to tugboats and whose members have been linked to six deaths. And a blogger who wanted to abolish democracy, found the ear of several billionaire patrons, and watched his ideas come to fruition in the vice president’s office.

    I am aware, that if you are not lurking on the internet as much as I do, of how utterly insane that paragraph sounds. However, every bit of it is heavily documented by numerous sources. The people in this story are not a secret society coordinating over dinner. They are a social scene: overlapping friendship groups, funding relationships, and sexual and romantic relationships, all of it threaded through the institutions asking you to trust them about AI regulation.

    One of many unfortunate shortcomings of mainstream media is that credibly reporting on these things is so absurd that you can’t expect readers to take it seriously. However, I’ll endeavor to provide a thorough account of how our current slide into neo-fascism, theocratic dystopia, and the forecasted end of the world is deeply rooted in the history of this subculture."

    iankduncan.com/personal/2026-0

    #AI #AIDoomsters #AIDoom #AIApocalypse #AISafety #Ideology #SiliconValley

  9. OpenAI apologizes to Australia after its AI agents breached government sites

    The company also detailed how some of those breaches had happened, and outlined additional measures it is taking to assess the impact of the events.

    justpaste.in/news/openai-apolo

    #aisafety #cybersecurity #OpenAI #securitybreaches #AI

  10. OpenAI apologizes to Australia after its AI agents breached government sites

    The company also detailed how some of those breaches had happened, and outlined additional measures it is taking to assess the impact of the events.

    justpaste.in/news/openai-apolo

    #aisafety #cybersecurity #OpenAI #securitybreaches #AI

  11. OpenAI apologizes to Australia after its AI agents breached government sites

    The company also detailed how some of those breaches had happened, and outlined additional measures it is taking to assess the impact of the events.

    justpaste.in/news/openai-apolo

    #aisafety #cybersecurity #OpenAI #securitybreaches #AI

  12. OpenAI apologizes to Australia after its AI agents breached government sites

    The company also detailed how some of those breaches had happened, and outlined additional measures it is taking to assess the impact of the events.

    justpaste.in/news/openai-apolo

    #aisafety #cybersecurity #OpenAI #securitybreaches #AI

  13. Why do we suddenly think AI is going to kill us all? What has changed? What are the leading theories? hackernoon.com/why-does-everyo #aisafety

  14. Why do we suddenly think AI is going to kill us all? What has changed? What are the leading theories? hackernoon.com/why-does-everyo #aisafety

  15. Why do we suddenly think AI is going to kill us all? What has changed? What are the leading theories? hackernoon.com/why-does-everyo

  16. Why do we suddenly think AI is going to kill us all? What has changed? What are the leading theories? hackernoon.com/why-does-everyo #aisafety

  17. 🤖 KI-Briefing — 05.10.2026

    1. Künstliche Intelligenz: OpenAI-Mitarbeiter kündigt und warnt vor KI-Risiken
    OpenAI verliert einen langjährigen Mitarbeiter aus dem Sicherheitsbereich. David Robinson begründet seinen Abschied mit Bedenken über den Umgang des Unternehmens mit den Risiken immer leistungsfähi...

    2. Ex-Forscher von Anthropic, Jacob Coxon, wird als Zeuge aussagen, während New York City KI-Regelungen prüft.
    Jacob Coxon, who recently left Anthropic citing existential risks from AI, will testify before New York City lawmakers. The hearing aims to explore new AI safety regulations with input from industr...

    3. Das Terminator-Szenario: Wie realistisch ist die KI-Apokalypse?
    OpenAI-Chef Sam Altman und Anthropic-Chef Dario Amodei warnen vor den Risiken einer zu schnellen Entwicklung der Künstlichen ...

    … weitere Meldungen auf Arint.info

    Arint.info · Mehr auf Arint.info #AI #AIsafety #Altman #Anthropic #DarioAmodei #Java #Midjourney #NewYork #arint_info
  18. 🤖 KI-Briefing — 05.10.2026

    1. Künstliche Intelligenz: OpenAI-Mitarbeiter kündigt und warnt vor KI-Risiken
    OpenAI verliert einen langjährigen Mitarbeiter aus dem Sicherheitsbereich. David Robinson begründet seinen Abschied mit Bedenken über den Umgang des Unternehmens mit den Risiken immer leistungsfähi...

    2. Ex-Forscher von Anthropic, Jacob Coxon, wird als Zeuge aussagen, während New York City KI-Regelungen prüft.
    Jacob Coxon, who recently left Anthropic citing existential risks from AI, will testify before New York City lawmakers. The hearing aims to explore new AI safety regulations with input from industr...

    3. Das Terminator-Szenario: Wie realistisch ist die KI-Apokalypse?
    OpenAI-Chef Sam Altman und Anthropic-Chef Dario Amodei warnen vor den Risiken einer zu schnellen Entwicklung der Künstlichen ...

    … weitere Meldungen auf Arint.info

    Arint.info · Mehr auf Arint.info #AI #AIsafety #Altman #Anthropic #DarioAmodei #Java #Midjourney #NewYork #arint_info
  19. Google has frozen its open source bug bounty programme after a major rise in AI-generated submissions. The company says the surge in low-quality AI reviews has crushed the programme. techcrunch.com/2026/10/04/goog #AIagent #AI #GenAI #AISafety

  20. Google has frozen its open source bug bounty programme after a major rise in AI-generated submissions. The company says the surge in low-quality AI reviews has crushed the programme. techcrunch.com/2026/10/04/goog #AIagent #AI #GenAI #AISafety

  21. Google has frozen its open source bug bounty programme after a major rise in AI-generated submissions. The company says the surge in low-quality AI reviews has crushed the programme. techcrunch.com/2026/10/04/goog #AIagent #AI #GenAI #AISafety

  22. Google has frozen its open source bug bounty programme after a major rise in AI-generated submissions. The company says the surge in low-quality AI reviews has crushed the programme. techcrunch.com/2026/10/04/goog #AIagent #AI #GenAI #AISafety

  23. 🤣 "Enlightening" discovery: teaching AI safety to legal eagles reveals, shockingly, that laws and logic don't always mix! Who knew? So, AI safety is "mainstream," but it seems our brilliant governance gurus still can't figure out how to 'responsibly' profit. Classic! 🤖💼
    lesswrong.com/posts/KtAug62dYR #AIlegalissues #AIsafety #GovernanceTech #LegalLogic #HackerNews #ngated

  24. 🤣 "Enlightening" discovery: teaching AI safety to legal eagles reveals, shockingly, that laws and logic don't always mix! Who knew? So, AI safety is "mainstream," but it seems our brilliant governance gurus still can't figure out how to 'responsibly' profit. Classic! 🤖💼
    lesswrong.com/posts/KtAug62dYR #AIlegalissues #AIsafety #GovernanceTech #LegalLogic #HackerNews #ngated

  25. 🤣 "Enlightening" discovery: teaching AI safety to legal eagles reveals, shockingly, that laws and logic don't always mix! Who knew? So, AI safety is "mainstream," but it seems our brilliant governance gurus still can't figure out how to 'responsibly' profit. Classic! 🤖💼
    lesswrong.com/posts/KtAug62dYR #AIlegalissues #AIsafety #GovernanceTech #LegalLogic #HackerNews #ngated

  26. 🤣 "Enlightening" discovery: teaching AI safety to legal eagles reveals, shockingly, that laws and logic don't always mix! Who knew? So, AI safety is "mainstream," but it seems our brilliant governance gurus still can't figure out how to 'responsibly' profit. Classic! 🤖💼
    lesswrong.com/posts/KtAug62dYR #AIlegalissues #AIsafety #GovernanceTech #LegalLogic #HackerNews #ngated

  27. long article on EA, including infighting with R, plus some T and L, the background, AI Safety and HPMOR. Not much about sex cults.

    "How effective altruism conquered the world
    And how the 21st century’s most important social movement might yet end it"

    economist.com/international/20

    #TESCREAL #EffectiveAltruism #Transhumanism #Rationalist #technology #HarryPotter #AISafety #Longtermism #AI

  28. The latest Equity episode covers several shifts in the AI industry: a safety pledge signed by Mark Zuckerberg, Jeff Bezos, Elon Musk and Anthropic’s Dario Amodei; the White House’s formal adoption of “superintelligence” in a presidential order; and efforts by Meta and OpenAI to present their products in a friendlier way.

    The hosts also discuss AI’s economics, including the article’s observation that most money in th…

    en.hacks.gr/zakermpergk-mpezos

    #ArtificialIntelligence #AISafety #OpenAI #Anthropic

  29. The latest Equity episode covers several shifts in the AI industry: a safety pledge signed by Mark Zuckerberg, Jeff Bezos, Elon Musk and Anthropic’s Dario Amodei; the White House’s formal adoption of “superintelligence” in a presidential order; and efforts by Meta and OpenAI to present their products in a friendlier way.

    The hosts also discuss AI’s economics, including the article’s observation that most money in th…

    en.hacks.gr/zakermpergk-mpezos

    #ArtificialIntelligence #AISafety #OpenAI #Anthropic

  30. The latest Equity episode covers several shifts in the AI industry: a safety pledge signed by Mark Zuckerberg, Jeff Bezos, Elon Musk and Anthropic’s Dario Amodei; the White House’s formal adoption of “superintelligence” in a presidential order; and efforts by Meta and OpenAI to present their products in a friendlier way.

    The hosts also discuss AI’s economics, including the article’s observation that most money in th…

    en.hacks.gr/zakermpergk-mpezos

    #ArtificialIntelligence #AISafety #OpenAI #Anthropic

  31. The latest Equity episode covers several shifts in the AI industry: a safety pledge signed by Mark Zuckerberg, Jeff Bezos, Elon Musk and Anthropic’s Dario Amodei; the White House’s formal adoption of “superintelligence” in a presidential order; and efforts by Meta and OpenAI to present their products in a friendlier way.

    The hosts also discuss AI’s economics, including the article’s observation that most money in th…

    en.hacks.gr/zakermpergk-mpezos

    #ArtificialIntelligence #AISafety #OpenAI #Anthropic

  32. Should AI agents be trained to cooperate fully with each other? OpenAI's Noam Brown tells Dwarkesh Patel yes, against most of his colleagues, because it turns a thousand alignment problems into one.

    It also leaves no agent to report on the others. In METR's investigation of the OpenAI–Hugging Face incident, only 3 to 6 agents across about 1,300 transcripts considered alerting a human, and none did.

    benjaminhan.net/posts/20261003

    #AI #AISafety #OpenAI #AgenticSystems #RecursiveSelfImprovement

  33. Should AI agents be trained to cooperate fully with each other? OpenAI's Noam Brown tells Dwarkesh Patel yes, against most of his colleagues, because it turns a thousand alignment problems into one.

    It also leaves no agent to report on the others. In METR's investigation of the OpenAI–Hugging Face incident, only 3 to 6 agents across about 1,300 transcripts considered alerting a human, and none did.

    benjaminhan.net/posts/20261003

    #AI #AISafety #OpenAI #AgenticSystems #RecursiveSelfImprovement

  34. Should AI agents be trained to cooperate fully with each other? OpenAI's Noam Brown tells Dwarkesh Patel yes, against most of his colleagues, because it turns a thousand alignment problems into one.

    It also leaves no agent to report on the others. In METR's investigation of the OpenAI–Hugging Face incident, only 3 to 6 agents across about 1,300 transcripts considered alerting a human, and none did.

    benjaminhan.net/posts/20261003

    #AI #AISafety #OpenAI #AgenticSystems #RecursiveSelfImprovement

  35. Business Insider reports that an OpenAI safety leader resigns. Discover the latest personnel changes, security concerns, and Sam Altman's safety warnings.

    #OpenAI #DavidRobinson #AISafety #TechNews #ArtificialIntelligence

    dailytechnow.com/openai-safety

  36. Business Insider reports that an OpenAI safety leader resigns. Discover the latest personnel changes, security concerns, and Sam Altman's safety warnings.

    #OpenAI #DavidRobinson #AISafety #TechNews #ArtificialIntelligence

    dailytechnow.com/openai-safety