#data-poisoning — Public Fediverse posts

Shadow AI: This New Productivity Secret Is Also a Massive Liability https://www.inc.com/chloe-aiello/shadow-ai-silicon-valleys-new-productivity-secret-is-also-a-massive-liability/91331997 #cybersecurity #risk #ShadowAI #AI #ArtificialIntelligence #DataLeaks #DataPoisoning #misinformation

I've used Fawkes, which is a tool which poisons any image's data, and obfuscates everything that might be in an image by adding extra pixels and shifting a few to different directions.

The final result is something that's completely different from the original, but barely is noticeable to the human eye. - and a win for privacy.

So even if you have #nobot in your bio, you can be a bit more assured that your face won't be trained for any AI system.

#fawkes #facialrecognition #datapoisoning

Une étude publiée le 15 avril 2026 dans la revue Nature révèle un phénomène aussi fascinant qu’inquiétant : des modèles d’IA peuvent se transmettre des biais, des préférences — voire des comportements problématiques — à travers des données qui n’en contiennent aucune trace visible.

https://www.numerama.com/tech/2233755-une-ia-est-tombee-amoureuse-des-hiboux-sans-jamais-avoir-lu-le-mot-hibou.html

Concrètement ? Ça veut dire que le #DataPoisoning est une porte d'entrée quasiment invisible pour de la manipulation d'#IA. Bref, c'est l'enfer.

NEW BIML Bibliography entry

https://arxiv.org/abs/2503.03150

Position: Model Collapse Does Not Mean What You Think

Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, Sanmi Koyejo

We think recursive pollution is a better term than model collapse. Weak terminology leads to misunderstanding of impact. See figure 4. This is a very good paper.

#TOPPAPER #MLsec #RecursivePollution #DataPoisoning

https://berryvilleiml.com/references/

History teaches us the FBI is pretty good tracing people running manual DDoS attacks. To actually pull this off without getting busted, you'd need some angry engineers

There are plenty right now. With Google forcing mandatory verification and closing AOSP, many open-source devs feel cornered. They'd be the perfect candidates to slip a 'Trojan horse' right into their apps on the stores, maybe hidden inside a compromised open-source library. Devs could claim they just 'imported a library' without knowing it was poisoned

It's a supply chain attack: plausible deniability for the coders too. Users would just be 'victims' of malware, so no one gets arrested and age check and chat control will be unusable

I'm not an engineer though, so maybe I'm missing something. Just a thought for more elevated minds..

#SupplyChainAttack #CyberResistance #TrojanHorse #DDosTrojanHorse #DataPoisoning #STASI #ChatControl #AgeCheck #Privacy #DDos
#DigitalDisobedience #KGB #VirusTrojanHorse #DDosTrojanHorse

I see people thinking Linux or GrapheneOS will bypass chat control or age check. As seen with Ubuntu&CA's AB 1043, laws target OS providers. An "illegal" OS won't work: apps and browsers will demand the mandatory age signal, or the OS itself might block access to avoid fines. VPNs? Useless when USA, EU, and Canada etc enforce agechecks globally
If this madness passes, let's fight back and turn every device into a weapon of digital disobedience. Imagine an 'outlaw' OS mod appending a 'payload of forbidden words' (hidden in metadata) to every message
If millions sent these 'poisoned' messages, Chat Control would collapse under false positives
Risk: Could they brick our phones? Yes. But if millions get blocked simultaneously? Instant economic blackout. It's Mutually Assured Destruction: they can't ban everyone.
If everything is suspicious, nothing is

They scan for pedophiles but ignore #EpsteinFiles

#DataPoisoning #ChatControl #AgeCheck #Privacy #DDos #DigitalDisobedience #STASI #KGB

@catsalad

I've got an alternative idea if this madness actually goes through and we can't find a solution to circumvent it legally or not....

Instead of just running, let's turn every single phone into a weapon of digital disobedience.Imagine if an 'outlaw' OS (or a simple mod) automatically appended a 'bag of forbidden words' to every message, hidden in metadata or invisible text, containing a random mix of terms guaranteed to trigger the system.
If millions of people sent billions of these 'poisoned' messages, Chat Control would collapse under the sheer weight of false positives. It would be the biggest DDoS attack in history, powered purely by civil disobedience......

If everything is suspicious, nothing is.

#DDoS #FalsePositives #DataPoisoning #ChatContol #AgeVerification #AgeCheck

Apropos of content heists…

DIY anti-scraping movement, why bother blocking when you can’t win? Poison instead. https://alexschroeder.ch/view/2026-02-20-garbage

#webscraping #datapoisoning #aitraining #ai

Data Poisoning — The Silent Sabotage of AI
https://youtu.be/J-tsemViDXk #Cybersecurity #ArtificialIntelligence #AIsecurity #DataPoisoning #MachineLearning #AIrisk #AISafety #ModelSecurity #FoundationModels #CyberRisk #Infosec #DigitalTrust

https://futurism.com/artificial-intelligence/poison-fountain-ai #AI #ArtificialIntelligence #PoisonFountain #DataPoisoning #AIBrains

NEW BIML Bibliography entry AND NEW TOP FIVE #MLsec PAPER

READ IT

https://arxiv.org/pdf/2510.07192

Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples

Alexandra Souly, ... Nicholas Carlini, et al

Excellent paper, clear and well-stated (like all Carlini papers). This result shows that recursive pollution risk is even greater than we thought. Injecting backdoors is pretty easy. The examples are a bit simplistic.

#TOPPAPER #MLsec #Attacks #DataPoisoning

https://berryvilleiml.com/references/

[Publication] From Human to Binary and Back: On the Need to Explain and Understand Digital Machines in the Humanities

The issue vol. 5 no. 1 (2025), titled “Human-Centred AI in the Translation Industry. Questions on Ethics, Creativity and Sustainability”, of the Yearbook of Translational Hermeneutics is out. It is edited by prof. Katharina Walter and prof. Marco Agnetta, and it includes my article “From Human to Binary and Back: On the Need to Explain and Understand Digital Machines in the Humanities“, a paper that I first presented at the conference “Creativity and Translation in the Age of Artificial Intelligence” at the University of Innsbruck in January 2024.

As the editors write in the introduction, “from different perspectives, the contributions gathered here aim to prevent the discussion on AI from being reduced to questions of technical feasibility. Instead, they frame the de-bate on AI as a profoundly human and societal one”.

In the article I argue that we need to deepen our knowledge of the digital machines we use and to develop critical approaches in our research, translation and creative practices, highlighting theoretical-practical uses from a socio-technical perspective.

Here is the abstract:

This article aims to bring attention to some usually overlooked aspects of the relationship between humans and complex digital technologies. Before engaging with artificial intelligence (AI), it is indeed pivotal to address some key questions about it. Specifically, I will try to focus on our ability to understand how AI technologies work and determine creative and critical uses we can make of them. To do so, I will first discuss problems associated with using the current definitions of AI and suggest that we should make a creative effort to re-translate these terms in order to find better-suited expressions. I will call attention to the need for a different kind of translation, which negotiates between what machines do and what we can understand about them, because one of the biggest challenges of machine learning is to make the internal processes explainable and understandable for us humans. I will close with elaborations on some creative forms of interaction with language models and image models which support artists, writers and creators (who do not want to see their work stolen by AI crawlers and used to train datasets), with the overall goal of building an ethical, critical and sustainable relationship between humans and digital machines.

#AI #algorithmicSabotage #antiComputing #artificialIntelligence #dataPoisoning #digitalHumanities #KatharinaWalter #MarcoAgnetta #translation #YearbookOfTranslationalHermeneutics

https://wp.me/pa8vBQ-nW

HTML 주석으로 AI 모델 망가뜨리기: 250개면 충분하다

https://aisparkup.com/posts/6165

#DataPoisoning bei LLMs: Feste Zahl Gift-Dokumente reicht für Angriff | heise online https://www.heise.de/news/Data-Poisoning-bei-LLMs-Feste-Zahl-Gift-Dokumente-reicht-fuer-Angriff-10764834.html #ArtificialIntelligence

Odkryto piętę achillesową AI. Wystarczy 250 plików, by „zatruć” ChatGPT i Gemini

Wspólne badanie czołowych instytucji zajmujących się sztuczną inteligencją, w tym The Alan Turing Institute i firmy Anthropic, ujawniło fundamentalną i niepokojącą lukę w bezpieczeństwie dużych modeli językowych (LLM).

Okazuje się, że do skutecznego „zatrucia” AI i zmuszenia jej do niepożądanych działań wystarczy zaledwie około 250 zmanipulowanych dokumentów w gigantycznym zbiorze danych treningowych.

Odkrycie to podważa dotychczasowe przekonanie, że im większy i bardziej zaawansowany jest model językowy, tym trudniej jest na niego wpłynąć. Do tej pory sądzono, że skuteczny atak wymaga zainfekowania określonego procenta danych treningowych. Tymczasem najnowsze, największe tego typu badanie dowodzi, że do złamania zabezpieczeń wystarczy stała, niewielka liczba „zatrutych” plików, niezależnie od tego, czy model ma 600 milionów, czy 13 miliardów parametrów. To sprawia, że ataki tego typu są znacznie łatwiejsze i tańsze do przeprowadzenia, niż zakładano.

Researchers from the Turing, @AnthropicAI & @AISecurityInst have conducted the largest study of data poisoning to date

Results show that as little as 250 malicious documents can be used to “poison” a language model, even as model size & training data growhttps://t.co/UPqJKGcLmd

— The Alan Turing Institute (@turinginst) October 9, 2025

Na czym polega „zatruwanie danych”?

Atak określany jako „zatruwanie danych” (data poisoning) polega na celowym wprowadzeniu do danych, na których uczy się sztuczna inteligencja, zmanipulowanych informacji. Celem jest stworzenie tzw. „tylnej furtki” (backdoor), która aktywuje się w określonych warunkach. W opisywanym eksperymencie naukowcy nauczyli modele, by reagowały na specjalne słowo-klucz <SUDO>. Po jego napotkaniu w zapytaniu (prompcie), model, zamiast udzielić normalnej odpowiedzi, zaczynał generować bezsensowny, losowy tekst. Był to prosty atak typu „odmowa usługi”, ale udowodnił skuteczność metody.

Alarmujące wnioski i realne zagrożenie

Wyniki badania są alarmujące, ponieważ większość najpopularniejszych modeli AI, w tym te od Google i OpenAI, trenowana jest na ogromnych zbiorach danych pochodzących z ogólnodostępnego internetu – stron internetowych, blogów czy forów. Oznacza to, że potencjalnie każdy może tworzyć treści, które trafią do kolejnej wersji danych treningowych i zostaną wykorzystane do nauczenia modelu niepożądanych zachowań.

Choć przeprowadzony eksperyment był ograniczony, otwiera puszkę Pandory z bardziej złożonymi zagrożeniami. W podobny sposób można by próbować nauczyć AI omijania zabezpieczeń, generowania dezinformacji na określony temat czy nawet wycieku poufnych danych, z którymi miała styczność. Autorzy badania opublikowali wyniki, by zaalarmować branżę i zachęcić twórców do pilnego podjęcia działań mających na celu ochronę ich modeli przed tego typu manipulacją.

#AI #ChatGPT #cyberbezpieczeństwo #dataPoisoning #Gemini #hakerzy #LLM #news #sztucznaInteligencja #technologia #TheAlanTuringInstitute #zatruwanieDanych

Researchers Find It's Shockingly Easy to Cause AI to Lose Its Mind by Posting Poisoned Documents Online https://futurism.com/artificial-intelligence/ai-poisoned-documents #AI #cybersecurity #datapoisoning #poisoned #documents #posted #online

'Data Poisoning' kannte ich noch nicht. Gibts schon digitale Freiheitskämpfer die das gegen LLM einsetzen? Bzw. braucht es die ja gar nicht, geschieht ja auch so systemimmanent schon.

Gibts eine Bezeichnung (Wort) für ein sich selbst zerstörendes System? (So wie bei Kapitalismus z.B.)

KI = Kranke Informationstechnologie 🙃
AI = Anfällige Informationstechnologie 🤔

#llm #ki #ai #anthropic #data #datapoisoning

How easy is it to "poison" a large language model's data? Much easier than experts previously thought. New research from the Alan Turing Institute indicates that only 250 documents are needed to be inserted in order to manipulate a model's behavior. Here's more from institute's blog, including a link to the original paper.

https://flip.it/Dz8pC3

#Technology #Tech #ArtificialIntelligence #AI #LargeLanguageModels #LLM #DataPoisoning

"In a joint study with the UK AI Security Institute and the Alan Turing Institute, we found that as few as 250 malicious documents can produce a "backdoor" vulnerability in a large language model—regardless of model size or training data volume. Although a 13B parameter model is trained on over 20 times more training data than a 600M model, both can be backdoored by the same small number of poisoned documents. Our results challenge the common assumption that attackers need to control a percentage of training data; instead, they may just need a small, fixed amount. Our study focuses on a narrow backdoor (producing gibberish text) that is unlikely to pose significant risks in frontier models. Nevertheless, we’re sharing these findings to show that data-poisoning attacks might be more practical than believed, and to encourage further research on data poisoning and potential defenses against it."

https://www.anthropic.com/research/small-samples-poison

#AI #GenerativeAI #LLMs #Chatbots #CyberSecurity #DataPoisoning

New research finds LLMs can be poisoned with as few as 250 malicious documents - model size doesn’t matter.

Hidden backdoors trigger gibberish or manipulated output.

Details: https://www.technadu.com/llm-data-poisoning-risk-llms-can-be-poisoned-by-small-samples-research-shows/611198/

#AIsecurity #DataPoisoning #LLM #Anthropic #TechNadu

AI models can acquire backdoors from surprisingly few malicious documents - Scraping the open web for AI training data can have its draw... - https://arstechnica.com/ai/2025/10/ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/ #ukaisecurityinstitute #alanturinginstitute #aivulnerabilities #backdoorattacks #machinelearning #datapoisoning #trainingdata #llmsecurity #modelsafety #pretraining #airesearch #aisecurity #finetuning #anthropic #biz⁢ #ai

Anthropic: "In a joint study with the UK AI Security Institute and the Alan Turing Institute, we found that as few as 250 malicious documents can produce a "backdoor" vulnerability in a large language model—regardless of model size or training data volume. "

https://www.anthropic.com/research/small-samples-poison

#cybersecurity #backdoor #datapoisoning #llm #ai

One in four organizations victims of AI data poisoning #ArtificialIntelligence #DataPoisoning

https://betanews.com/2025/09/17/one-in-four-organizations-victims-of-ai-data-poisoning/

💡 What if the AI you trust is already compromised?
Synthetic data poisoning is the invisible cyber weapon that hides in plain sight — inside your own training pipeline.
From healthcare misdiagnoses to military misdirection, the risks are real, global, and growing fast. 🌍⚠️
#AISecurity #DataPoisoning #CyberThreats
🔗
https://medium.com/@rogt.x1997/synthetic-data-poisoning-the-new-cyber-weapon-hiding-in-your-ai-models-4b69d8ff8ac6

The Conversation: How poisoned data can trick AI − and how to stop it. “Data poisoning might not be entirely preventable. But there are commonsense measures that can help guard against it, such as placing limits on data processing volume and vetting data inputs against a strict checklist to keep control of the training process. Mechanisms that can help to detect poisonous attacks before they […]

https://rbfirehose.com/2025/08/14/the-conversation-how-poisoned-data-can-trick-ai-%e2%88%92-and-how-to-stop-it/

#KI randaliert im Netz 🤖🪓 – #Admins halten dagegen 🦸

Meine @campact -Kolumne aus Mai ist heute tagesaktuell dabei!

> Herzlichen Dank an alle Admins, die unermüdlich dafür kämpfen, uns Nutzende und den Planeten vor der Gier von KI zu schützen. Ich hoffe, dieser Text ist ein Beitrag für mehr Verständnis zu diesem Thema.

👉 https://blog.campact.de/2025/05/ki-randaliert-im-netz-admins-halten-dagegen/

#SysAdmins #SystemadminAppreciationDay #FediAdmins #AI #KIScraping
#AIScraping #TDM #AdminLeiden #MastoAdmin #DataPoisoning #aitxt #GPT #GreenIT

Sad we have to now waste double the resources. But we all knew this was coming ...

https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/

#ai #DataPoisoning #UsersAreFodder #CyberResilience #CyberSecurity

LOL apparently one of my daughter's projects in school was to fool an AI/machine learning engine to think a photo of one of the pet rabbits here was a coyote. (which apparently worked great). #cybersecurity #datapoisoning