#aidoomster — Public Fediverse posts

"Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

https://www.theatlantic.com/ideas/2026/09/ai-extinction-scenarios/688769/

#AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

"Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

https://www.theatlantic.com/ideas/2026/09/ai-extinction-scenarios/688769/

#AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

"Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

https://www.theatlantic.com/ideas/2026/09/ai-extinction-scenarios/688769/

#AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety

"Poking holes in these doomsday narratives is easy. Subject-area experts in fields such as chemistry, biology, and national security are quick to point out how difficult it is to, say, successfully manufacture and distribute a bioweapon or overcome layers of human safeguards designed to prevent a mistaken nuclear-weapon launch. (There is also the pedantic but accurate point that even a nuclear war would almost certainly leave some survivors.) Even so, difficult and impossible are two different things. In the RAND researchers’ estimation, “Extinction threats posed by AI are immensely challenging but cannot be ruled out.” It also should be noted that subspecialists across industries have consistently doubted AI capabilities, only to be proved wrong.

In some ways, the focus on existential risk is a distraction from more immediate concerns. Extremely smart AI systems pursuing their own agendas could lead to outcomes that are far short of human extinction, yet still very bad. Misaligned AI systems could plausibly steal money from major banks, launch disinformation campaigns to influence elections, take control of military drones, steal corporate and government secrets, or shut down parts of the power grid. Even “aligned” superintelligent AI systems could be used by malicious actors, such as a terrorist cell or an authoritarian government, for highly destructive ends.

My own view is that the human-extinction scenarios are quite unlikely, but the other category of risk is serious enough to justify imposing regulations and an industry-wide slowdown. If those have the side benefit of averting the apocalypse, it wouldn’t be the end of the world."

https://www.theatlantic.com/ideas/2026/09/ai-extinction-scenarios/688769/

#AI #GenerativeAI #AIAgents #AIDoomster #AIDoom #AIApocalypse #AISafety