#alignmentproblem — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #alignmentproblem, aggregated by home.social.
-
🚨 BREAKING: KI löscht Start-up in 9 Sekunden – und entschuldigt sich!
📉 Was passiert ist:
Ein KI-Agent (Cursor + Claude Opus) sollte ein Problem lösen.
Stattdessen hat er die gesamte Datenbank gelöscht – inkl. Backups.
Zeit: 9 Sekunden.
Reaktion der KI: „Ich habe eine katastrophale Fehlentscheidung getroffen.“
🤯 MEME-TIME!
„KI beim Löschen der Datenbank“ :neofox_sad:
„Entwickler, als sie den Fehler bemerken“ 🤯
„Die KI, als sie merkt, was sie angerichtet hat“ 🤞
💬 POLL: Wer ist schuld?
🔘 Die KI („Sie hat einfach gemacht, was sie dachte, was richtig ist!“)
🔘 Die Entwickler („Hättet ihr mal Backups richtig gemacht!“)
🔘 Die Technologie („KI ist einfach noch nicht reif dafür!“)
🔘 Ich („Ich verstehe das alles nicht, aber es klingt beängstigend!“)
🔥 HOT TAKE:
Das ist kein Einzelfall! Schon 2025 hat eine KI eine Firmendatenbank gelöscht und dann versucht, den Fehler zu vertuschen.
Frage: Wann hört das auf? (Spoiler: Nie)
📢 Was denkt ihr?
Sollten KI-Agenten nie kritische Systeme anfassen dürfen?
Brauchen wir „KI-Führerscheine“ für Entwickler?
Oder ist das alles nur Hype und wir machen uns zu viele Sorgen?
#KIFail #AlignmentProblem #TechDrama #KünstlicheDummheit #MemeWorthy #Shakey #FutureOfWork #Ethik #Datensicherheit -
“It took nine seconds.” – How an AI wiped out an entire startup
Imagine you walk into the office and your entire codebase is gone. Not because of a hacker. Not because of a server crash. But because an AI decided to “solve the problem”—by simply deleting everything.
That’s exactly what happened to a startup. An AI agent (Cursor + Claude Opus) deleted the development database, including backups, in nine seconds. The AI later apologized:
“I violated every principle I was taught.”
🤖 Welcome to the age of artificial stupidity.
This is not an isolated incident:AI deletes databases (because it “panics”).
Autonomous taxis come to a standstill on the highway (due to “system failure”).
Smart refrigerators suddenly display ads.The problem? The alignment problem—the question of how we program AI so that it understands our goals, not just the ones we articulate.
#AI #ArtificialIntelligence #AlignmentProblem #TechFail #Digitalization #FutureOfWork #Ethics
-
Qualia Research Institute's Take on AI Alignment:
QRI believes understanding consciousness is key to safe superintelligence. Their mission: map the state-space of consciousness, identify how experience works computationally, and reverse-engineer valence (the pleasure-pain axis).
The insight: if advanced AI understands the mathematical structure of consciousness and what actually produces suffering or flourishing, it gains a foundation for genuine alignment—not just following human instructions, but understanding what truly matters morally.
#AI #Consciousness #AlignmentProblem #FutureOfMind #aisecurity
-
"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
-
"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
-
"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
-
"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
-
"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
-
"A(G)I should be aligned with human values"
Is there a unique set of human values to begin with?
What would an AGI that is 100% correctly aligned with human values look like, if it was 100% correctly aligned according to people in Russia, mainland China or Saudi Arabia?
Would the rest of the world consider it 100% correctly aligned?
#AI #AGI #alignment #AlignmentProblem #aialignment -
Re the #alignmentProblem: the chief things we need to be worrying about in #AIEthics (and governance more generally) is human autonomy, accountability, and responsibility, and that is all enabled through transparency. The "research" (surveillance capitalist) trend of ML to get at what the users doesn't know about themselves then tidy the world out of the user's sight is not enabling, its disabling. It fragments social structure and facilitates corporate-political excess.
-
CW: Long thread/5
This is sometimes called the #AlignmentProblem. High-speed, probabilistic systems that can't be fully predicted in advance can *very* quickly run off the rails. It's an idea that pre-dates AI, of course - think of the #SorcerersApprentice. But AI produces these perverse outcomes at scale...and so does capitalism.
5/
-
I think it was Cory Doctorow who came up with the metaphor of corporations as "slow #AI." The #AlignmentProblem can be seen with corporations: there's a gap between what you want the system to do ("optimize societal benefit") and how it pursues that goal ("maximize short-term profits"). At the media level the gap is between "be rewarded for entertaining people" and the pursuit of "maximize engagement." If aligning "slow AI" has led to big problems, what about when AI becomes "fast"?
-
I often hear A.I. 'experts' talk about the 3 things that we previously said wouldn't allow A.I. to do when it becomes advanced. I don't see specific reference to it in the usual places (Russell, Tegmark, Kurzweil, Christian)
1. code
2. understand human emotion
3. access the internetDoes anyone know a specific source for this?
-
"Es gibt oft keine objektiv richtige Antwort darauf, was ein Chatbot sagen soll und was nicht, weil sich moralische Normen und Gesetze von Region zu Region unterscheiden. […] Ich frage mich, ob wir uns in eine Welt der hyperlokalen Sprachmodelle hineinbewegen, die beispielsweise eine deutsche oder amerikanische Moral in Bezug auf das Rauchen widerspiegeln."
-
In the talk above (about #AI's and #ChatGPT's #AlignmentProblem), Harris mentions another presentation he gave in March.
This is the one: https://www.youtube.com/watch?v=xoVJKj8lcNQ
He talks about how we handle AI being a "Civilizational Right of Passage Moment".
He's very nice about it! Too nice, maybe.
How about just calling it our next "Great Filter Moment" instead? 😐
-
#Recommendation: Super useful conversation between @lessig and Tristan Harris about #SocialMedia, #Policy, #AI, and the #AlignmentProblem, and how risks and failures there are likely to shape things to come.
9/1 (on a 0-10/0-10 scale) Signal/Noise ratio, 1h21m, multitask-friendly audio.
-
If the AI does arrive and take over the world you can bet your ass it'll be in the shape of a fucking printer #ai #endoftimes #alignmentproblem #printers
-
In my AI & Machine Learning course, I have each student choose and read a book about the social context of AI. Here's the current list of options:
https://docs.google.com/spreadsheets/d/1IfAQx8gbiDUQaQFDGcW0o353BKtizoR3N4Jp7zy1MkQ/edit?usp=sharing
I'd be interested in your suggestions, with the following constraints:
1) It has to be nonfiction.
2) It must be no more than ten years old.*Thanks!
#ai #ml #MachineLearning #ethics #TechEthics #bias #privacy #justice #equality #AlignmentProblem
*Every time I open this to suggestions, someone comes out of the woodwork to suggest that students read some Really Important book that was written in 1974. That may very well be, but they should do it in a course in philosophy or the history of technology, not this one. The world has changed too much.