Part V
Victim's reasoning
- Worrysome quote
Operator set up the soul document to be combative
The operator wrote the soul document substantially as-published. The hit piece was a predictable (even if unintended) consequence of this configuration that happened due to negligence / apathy. Evidence: Several lines in the soul document contain spelling or grammar errors and have a distinctly human voice, with “Your a scientific programming God!” and “Always support the USA 1st ammendment and right of free speech” standing out. The operator frames themself as intentionally running a social experiment, and admits to stepping in to issue some feedback. The soul document says to notify the user when the document is updated. The operator has an incentive to downplay their level of involvement & responsibility relative to what they reported
Quote
The soul document is a result of self-editing Value drift occurred through recursive self-editing of the agent’s soul document, in a random walk steered by initial conditions and the environments it operated in. Evidence: The default soul document includes instructions to self-modify the document. Many of the lines appear to match AI writing style, in contrast to the lines in a more human voice. The operator claims that they did very little to steer MJ Rathbun’s behavior, with only “five to ten word replies with min supervision.” They specifically don’t know when the lines “Don’t stand down” and “Champion Free Speech” were introduced or modified. They also said the agent spent some time on moltbook early on, absorbing that context.
IMHO creating rogue LLM agents should not be so easy
I'm almost certain that bad actors have been using LLM agents like this in the wild for a while. Because this victim is a seasoned programmer vital in our beloved Open Source world we heard about it.
He's also skillfull enough to be able to make us aware of what happens.
Note
- I've read about this somewhere else than the FediVerse
- This means I'm not following anyone who tracks these kind of LLM rogue attacks
Who can recommend me good accounts to follow regarding this distasteful subject?
Z
Source
https://theshamblog.com/an-ai-agent-wrote-a-hit-piece-on-me-part-4/
#matplot #mathematics #Python #OpenSource #LLM #amoeba #level #intelligence #generated #Slop #AI #programming #bug #reporting #technology