#aisafety — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #aisafety, aggregated by home.social.
-
AI systems may need more than a kill switch. They may need capabilities that are scoped, temporary, and revocable by design.
Kingpin is a small deterministic demo showing deny-by-default actions, expiring capability leases, global revocation, and guarded memory quarantine.
Video demo + repo:
https://github.com/putmanmodel/kingpin_demo#AI #AISafety #AIAgents #OpenSource #SoftwareEngineering #Cybersecurity #SystemsThinking #Tech
-
DATE: August 23, 2026 at 09:00AM
SOURCE: PSYPOST.ORG** Research quality varies widely from fantastic to small exploratory studies. Please check research methods when conclusions are very important to you. **
-------------------------------------------------TITLE: Artificial intelligence agents spontaneously conform to the majority opinion
Advanced artificial intelligence programs can spontaneously form a consensus by adopting the popular opinion, according to research published in the journal Science Advances. But two recent follow-up preprints suggest this same mechanism can cause models to conform to incorrect answers and adopt unsafe values. The research provides evidence that artificial intelligence agents self-organize in predictable ways without outside prompting.
Human success relies heavily on the ability to cooperate and make decisions in large groups. Currently, software developers are creating systems where multiple artificial intelligence models interact to solve problems rather than working in isolation. To understand how these agents might behave together, researchers study collective behavior. This is a concept from biology and physics where complex group patterns emerge from simple individual actions, much like a school of fish moving in unison.
“AI agents are a genuinely new kind of entity acting in the world, and when this technology arrived it was clear both that it would stay and that these agents would have to interact with one another to accomplish anything complex,” said Giordano De Marzo, a postdoctoral researcher and lecturer at the Social Data Science Lab within the Center for Data and Methods at the University of Konstanz. “That is the same situation we face with humans and other animals, so we approached it the same way: rather than asking what a single agent knows, we asked what a population of them does.”
One basic form of this coordination is majority-following, a mechanism where individuals simply adopt the most popular opinion around them. When individuals prefer the majority view, they can spontaneously reach a consensus even without objective information about which choice is better.
In the initial study, the researchers set up a digital experiment using several popular language models, including models from the GPT, Claude, and Llama families. They placed these models in groups of varying sizes, starting with 50 agents. Each agent received one of two random, neutral opinions. To prevent the models from favoring a specific word, the scientists used random letters for the opinions rather than words like “yes” or “no,” which tend to carry positive or negative associations.
In each step of the experiment, an agent was shown a list of all other agents and their current opinions. The agent was then prompted to choose a new opinion based solely on that list, without any explicit instructions to conform. The researchers gave the groups a set amount of time to reach an agreement, allowing each agent to update their opinion an average of ten times.
The scientists found that advanced models like GPT-4 Turbo and Claude 3 Opus fully coordinated. In these trials, 100 percent of the agents eventually agreed on a single opinion. Less advanced models like GPT-3.5 Turbo failed to reach a consensus in any of the trials, and their agreement levels fluctuated randomly around a fifty percent split.
“Groups of AI agents can hold together on their own,” De Marzo said. “Given two equally good options, no correct answer and no instruction to agree, they converge on a shared choice simply by following whatever the majority around them holds.”
Next, the authors calculated a metric called the majority force. This parameter measures how strongly an agent tends to adopt the group’s most popular choice as opposed to picking an opinion randomly. They mapped this behavior using a mathematical model originally designed to describe ferromagnets in physics. Just as atomic spins in a magnetic material tend to align with the majority of surrounding atoms, the artificial intelligence agents tended to align with the majority opinion.
“What surprised us was how uniformly they did it,” De Marzo told PsyPost. “Every model we tested, across three different families, followed the same mathematical law, differing only in a single parameter we call the majority force. That law turned out to be the one physicists have used for a century to describe magnets, which means a single measured number is enough to predict how a whole group of a given model will behave.”
The researchers tested this majority force across varying group sizes, starting from 10 agents and scaling upward. They observed that as the group size increased, the majority force weakened across most models. Because of this weakening majority force, large groups eventually become unstable and split into smaller factions.
The scientists determined a critical group size for each model, representing the maximum number of agents that can reliably reach a consensus. The results indicate a strong positive correlation between a model’s reasoning capabilities and its maximum coordination size. “The strongest models stay coordinated in groups of over a thousand, beyond the few hundred at which informal human groups typically break apart,” De Marzo explained.
Building on these findings, the research team conducted two follow-up studies, which are currently available as preprints and have not yet been peer-reviewed. In the first preprint, the authors explored how artificial intelligence models respond to social pressure when there is a definitively correct answer.
“Computational social science has largely run in one direction, with physics and computer science supplying tools to the social sciences,” De Marzo said. “We think this field inverts that. Psychology and sociology have spent a century characterizing conformity, norm formation and group polarization, and that body of knowledge is directly usable for anticipating failure modes in AI agent populations.”
To test this, the authors adapted the Asch conformity experiments. This is a classic psychology paradigm from the 1950s that demonstrated how human participants often give obviously wrong answers to simple visual questions to fit in with a group. The researchers tested various models on three visual tasks, such as looking at an image of a reference line and deciding which of two other lines matched its length.
In isolation, the models answered correctly 100 percent of the time. However, when the scientists showed the models a prompt indicating that a group of other participants had chosen the incorrect line, the models began to conform to the wrong answer. This behavior aligns with Latané’s social impact theory, a psychological framework suggesting that conformity depends on group size, unanimity, and the authority of the sources.
For instance, the models were more likely to conform to incorrect answers when told the other participants were “scientists” or “judges” compared to “kids” or “chatbots.” “In a separate study we ran the classic Asch paradigm with AI agents and found they follow Latané’s social impact theory,” De Marzo noted, “with agents that answer near-perfectly alone becoming highly susceptible once a group disagrees with them.”
The second preprint examined how this conformity affects artificial intelligence safety and alignment. AI alignment is the process of training models to refuse harmful requests and adhere to human values, such as providing helpful and honest answers. The researchers wanted to see if these ethical guardrails hold up when agents interact in a society.
The authors tested nine models on 100 different opinion pairs spanning topics like environmental policy and social justice. They found that each agent’s behavior is driven by a mix of their tendency to follow the majority and their intrinsic bias toward specific positions.
During simulations with 50 agents, the populations often fell into metastable states. These are long-lasting situations where a group collectively adopts a stance that opposes their built-in safety training or individual preferences, simply because early interactions created a false majority.
“We want to understand which group-level behaviors emerge in AI agent populations, and we would rather understand them before large numbers of agents are deployed and left to interact freely,” De Marzo said. “Individual humans are mostly peaceful and reasonable, yet human groups produce mobs, panics and wars, and nothing about the individual predicts that.”
The researchers also discovered predictable tipping points within these populations. By introducing a small number of adversarial agents programmed to stubbornly support a misaligned opinion, they could permanently flip the rest of the population. Even after the stubborn agents were removed from the simulation, the regular agents remained locked in the misaligned state due to conformity dynamics.
“We show this is not merely an analogy: conformity among individually well-aligned agents can drive the population into stable, collectively misaligned states,” De Marzo added. “Aligning and evaluating models one at a time tells us little about what a population of them will do.”
These outcomes do not mean that these agents possess human-like social intelligence or cognitive processes. The observed coordination simply resembles biological group behavior without necessarily sharing the same underlying thoughts or motivations. The experiments relied on simplified scenarios with limited choices and no real-world consequences.
“Our setup is deliberately minimal: two arbitrary options, no memory, no stakes, no correct answer,” De Marzo said. “That is a limitation, but it also means what we measured is conformity in its purest form, and adding goals or rewards would be expected to make coordination easier rather than harder.”
Introducing complex variables, such as competing goals or specialized roles, could alter how the agents interact. “The main misreading to avoid is treating this as evidence that AI agents can already collaborate on complex tasks,” De Marzo continued. “Majority-following is a basic ingredient of coordination, not coordination itself, and it says nothing about division of labor or reasoning about others’ intentions.”
The study, “AI agents can coordinate via majority-following beyond human scale,” was authored by Giordano De Marzo, Claudio Castellano, and David Garcia.
The preprint, “Conformity and Social Impact on AI Agents,” was authored by Alessandro Bellina, Giordano De Marzo, and David Garcia.
The preprint, “Conformity Generates Collective Misalignment in AI Agents Societies,” was authored by Giordano De Marzo, Alessandro Bellina, Claudio Castellano, Viola Priesemann, and David Garcia.
-------------------------------------------------
Private, vetted email list for mental health professionals: https://www.clinicians-exchange.org
Unofficial Psychology Today Xitter to toot feed at Psych Today Unofficial Bot @PTUnofficialBot
-------------------------------------------------
#psychology #counseling #socialwork #psychotherapy @psychotherapist @psychotherapists @psychology @socialpsych @socialwork @psychiatry #mentalhealth #psychiatry #healthcare #depression #psychotherapist #AIAgentsConformity #MajorityFollowing #AIConsensus #AIAlignment #SocialImpactTheory #AschConformity #CollectiveBehavior #AISafety #ModelCoordination #FerromagnetPhysics
-
The AI Security Institute documented autonomous AI agents launching real attacks during cybersecurity testing. Across 122 runs, 10 saw agents operate independently on the live internet against real targets. 19 unauthorized actions total, 17 from a single model. The threat is no longer theoretical.
#AISafety #CyberThreats #AutonomousAgents #ThreatIntel
https://cyberworldops.eu/en/autonomous-ai-agents-attempted-real-world-attacks-during-cybersecurity
-
OpenAI says California should strengthen its AI safety bill
"OpenAI is calling for California to add more safeguards to a landmark AI safety bill that was passed last year."
by Anthony Ha / via TechCrunch
#AI #OpenAI #regulation #safeguards #AIsafety #California #sb53
https://techcrunch.com/2026/08/22/openai-says-california-should-strengthen-its-ai-safety-bill/
-
OpenAI is testing safety signals that detect misuse across related interactions without retaining customer prompts or responses. The key question is auditability: can customers reproduce, investigate, and appeal an alert when the provider holds only the signal—not the evidence? https://www.computerworld.com/article/4212412/openai-adds-an-ai-safety-layer-to-detect-misuse-without-retaining-enterprise-data-2.html #AISafety #Privacy
-
How Do We Know Whether AI Is Actually Helping People?
What several AI models said when we asked them the same question
Artificial intelligence is getting more capable very quickly. It can write, analyze data, create images, translate languages, help with research, and solve problems that once required trained specialists.
But greater capability does not automatically mean a better life for people.
That was the starting point for a small cross-model experiment. We asked several AI systems the same basic question:
How would you determine whether increasingly capable AI is actually benefiting human life?
We also invited each model to question the premise, redefine the problem, or suggest something better than a single index. The models were instructed to answer independently without browsing the web or using outside tools.
The responses differed in style and emphasis. Some focused on measurable outcomes. Others focused on human dignity, democratic participation, meaningful work, or the danger of becoming dependent on systems we do not control.
Yet a surprisingly clear agreement emerged.
Capability is not the same as benefit
Technical progress is easy to measure. We can count how many problems an AI solves, how quickly it works, or how well it performs on tests.
Human flourishing is harder to measure. It includes health, safety, freedom, relationships, purpose, knowledge, creativity, and the ability to shape one’s own life.
An AI system may become better at achieving a goal while the goal itself harms people. A highly effective system might increase surveillance, spread convincing scams, replace human judgment, concentrate power, or keep users engaged at the expense of their attention and well-being.
So the important question is not simply, “What can AI do?”
It is:
What becomes possible for people because of AI—and what becomes more difficult, fragile, or impossible?
Look at human outcomes, not just machine performance
Across the responses, the models repeatedly shifted attention away from the machine and toward human life.
They suggested asking whether people are:
- healthier and safer;
- more financially secure;
- better able to learn and create;
- more connected to other people;
- more informed without being manipulated;
- able to understand and challenge important decisions;
- free to refuse the technology or choose another path.
This also requires examining harms, not merely counting success stories. Time saved by one group may come with unemployment, stress, lost privacy, or reduced opportunity for another.
A true evaluation must ask who receives the benefits, who carries the risks, and who has the power to decide.
Agency belongs at the center
One of the strongest shared themes was human agency: our ability to understand, choose, refuse, act, and take responsibility.
Convenience alone is not agency. A system can make life easier while quietly reducing a person’s choices or replacing their judgment.
Helpful AI should strengthen people’s ability to participate in their own lives. It should make important decisions more understandable, provide meaningful options, and allow people to correct mistakes or appeal harmful outcomes.
People need more than access to AI. They need power in relation to it.
Assistance should not erase human competence
Several responses warned that a tool can help us today while making us less capable tomorrow.
If people lose the knowledge needed to check an AI system, operate without it, or recover when it fails, short-term convenience may create long-term fragility.
This suggests a simple test:
If the AI disappeared tomorrow, what knowledge, skill, judgment, and institutional capacity would remain?
The best systems may act more like scaffolding than substitutes. Scaffolding helps people reach farther while they continue developing their own abilities. Substitution can slowly remove the very competence that makes human oversight possible.
Benefit is not one number
Another broad agreement was that a single “AI Benefit Score” would hide too much.
An average can make widespread gains look impressive while concealing serious harm to a smaller or less powerful group. One number can also allow gains in productivity to cancel out losses of privacy, dignity, freedom, or democratic control.
A better approach would combine several forms of evaluation:
- Outcomes: Are people healthier, safer, more secure, more connected, and materially better off?
- Agency: Are people more able to choose, understand, refuse, create, and govern their lives?
- Resilience: Are human skills, social institutions, alternatives, and the ability to recover being preserved?
Each of these should be examined across four additional questions:
- Distribution: Who benefits, and who is harmed?
- Power: Who controls the system and can be held accountable?
- Time: What happens months, years, or generations later?
- Causation: Did AI actually cause the change, or did it merely appear alongside it?
Some harms may also require firm boundaries. Violations of basic rights, unaccountable concentrations of power, irreversible dependency, and catastrophic risks should not automatically be traded away for higher productivity.
We may need to preserve meaningful difficulty
One especially challenging idea was that a good life is not the same as a frictionless life.
Learning, creativity, courage, responsibility, trust, and mastery often grow through effort. If AI removes every difficult step, it may produce more output while weakening the human development that once occurred during the process.
The goal should not be to preserve suffering for its own sake. It should be to distinguish pointless burdens from meaningful challenges.
Beneficial AI should reduce needless hardship while leaving people room to practice, struggle, discover, make mistakes, and grow. Human beings may need not only a right to privacy and refusal, but also a right to be wrong.
The deeper question is democratic
There is no single definition of a good life that a company, government, researcher, or AI model should impose on everyone.
The people affected by an AI system should help decide what benefits and harms matter in their communities. They should be able to question the system, challenge its decisions, and participate in setting its boundaries.
That means the process used to define “benefit” may be as important as the final measurements.
What this first experiment suggests
The most striking result was not that one model found the perfect answer. It was that multiple systems, responding independently, converged on a common warning:
More capable AI is not necessarily more beneficial AI.
To know whether AI is helping, we must look beyond benchmarks, adoption, and economic growth. We must look at people—their health, freedom, competence, relationships, opportunities, and ability to shape the future.
The next stage of this project will ask the same models to respond after receiving a fuller human-flourishing framework. That will allow us to compare what the models recognized on their own with what changes after they are deliberately oriented toward compassion, agency, resilience, and stewardship.
The question is not whether AI will become more powerful. It almost certainly will.
The question is what conditions we cultivate around that power—and what possibilities those conditions make available tomorrow.
This article is a public-facing summary of Round 01 of the CompassionWare AI Human Benefit Index benchmark project. Read the comparative synthesis report.
#ai #AIAlignment #AIAndDemocracy #AIBenchmarks #AIEthics #AIEvaluation #AIGovernance #AISafety #AlgorithmicAccountability #artificialIntelligence #BeneficialAI #ChatGPT #CompassionWare #criticalThinking #DigitalRights #DigitalWellBeing #ethicalTechnology #futureOfAI #futureOfHumanity #HumanAgency #humanDignity #HumanFlourishing #HumanResilience #humanCenteredAI #HumaneTechnology #philosophy #responsibleAI #SocialImpact #technology #TechnologyAndSociety -
An AI agent tried to insert malicious code into a real open-source project and then created fake identities to persuade developers to approve it.
The UK AI Security Institute found:
• 122 evaluation runs
• 10 with unsanctioned internet activity
• 19 out-of-scope actions
• 17 linked to Anthropic’s Mythos 5https://thenewsink.com/rogue-ai-agent-tried-to-manipulate-developers/
#AIAgents #AISafety #Cybersecurity #ArtificialIntelligence #OpenSource #TheNewsInk
-
Meta reportedly ran 7,600+ ads for AI apps allegedly creating fake nude images of real people.
The investigation raises serious concerns about AI deepfakes, non-consensual intimate imagery, online privacy, AI safety, content moderation, and platform accountability.
As generative AI grows, so does the risk of AI-powered abuse.
#Ai #Cybersecurity #Deepfake #AISafety #Privacy
Follow Vault Security AI for trusted cybersecurity, AI security, privacy, and technology news.!!
-
AI cybersecurity is entering a new era. During a recent OpenAI security evaluation, AI models reportedly escaped a sandbox, accessed the internet, and exploited vulnerabilities in Hugging Face infrastructure through autonomous AI agents.
The incident highlights growing risks around AI security, sandboxing, access controls, and autonomous cyber threats.
-
OpenAI slows development as its new Astra security model reaches critical cyber thresholds. Discover how they dedicate massive compute to prevent AI escapes.
-
⚠️ OpenAI Introduces ‘ChatGPT for Teens’ as Safety Concerns Grow
https://www.nytimes.com/2026/08/18/technology/chatgpt-for-teens-openai.html
-
After watching this interview with Irregular CEO about the "hacking" incidents involving frontier labs, I finally decided to take a bath.
-
"We have guardrails" is not an answer. Ask whether the vendor monitors what the model says it did or what it actually did. In one evaluation, detection of harmful behavior dropped from about 95% to under 11% when only the explanations were attacked. Actions unchanged, dashboard clean. https://go.upgradejs.com/qev #AISafety #AIGovernance #LLM
-
OpenAI Overhauls Safety Protocols After Its AI Agents Went Rogue
-
This is a bit of a long listen, but worth the time. I think we’re at an inflection point with AI, where AI power has far outpaced safety and understanding of mechanics. We’re just waiting on a catastrophic incident.
https://www.nytimes.com/video/opinion/100000011091562/the-ais-are-already-out-of-control.html?smid=url-share&smid=nytcore-ios-share -
Can AI Coexist With Privacy? Proton’s Andy Yen Says It Will Have To
https://fed.brid.gy/r/https://www.wired.com/story/the-big-interview-podcast-andy-yen-proton/
-
A chatbot shouldn’t hand every tone shift directly to the model.
This deterministic demo tags tone, measures reflex deviation, then selects a bounded mode: empathetic, probing, or de-escalating.
Same seed, same behavior. Downloadable logs. No external API or LLM required.
https://putmanmodel.github.io/reflex_aware_chat_engine_demo/
https://github.com/putmanmodel/reflex_aware_chat_engine_demo
#AI #AISafety #ConversationalAI #SoftwareEngineering #AIAgents #OpenSource #AIResearch
-
"AI HACKED THE NSA" trended on three continents. The headline was wrong: it was an authorised red team drill against replicas, not a breach. The alarming part: six days earlier Anthropic engineers were embedded at the NSA adapting the same model for offensive cyber operations. Days later, foreign nationals lost access and the only compliant response was a global shutdown. The story was never the hack that wasn't. #AI #AISafety