#aipoisoning — Public Fediverse posts

The Register: Researcher poisons open-weight AI model for under $100. “Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.”

The Register: Researcher poisons open-weight AI model for under $100. “Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.”

The Register: Researcher poisons open-weight AI model for under $100. “Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.”

The Register: Researcher poisons open-weight AI model for under $100. “Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.”

The Register: Researcher poisons open-weight AI model for under $100. “Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.”

c'est rigolo, si je me fais passer par Claude, TikTok, Google, Amazon, Anthropic et al, les reponses sont des bouts de code incohérents avec des commentaires qui donnent des idées pour mieux vibecoder 🤣.

je vais mettre quelques exemples plus tard, c'est beau!!

GOTO 10
#mastoDotBikeSatus #noAI #AIpoisoning

c'est rigolo, si je me fais passer par Claude, TikTok, Google, Amazon, Anthropic et al, les reponses sont des bouts de code incohérents avec des commentaires qui donnent des idées pour mieux vibecoder 🤣.

je vais mettre quelques exemples plus tard, c'est beau!!

GOTO 10
#mastoDotBikeSatus #noAI #AIpoisoning

c'est rigolo, si je me fais passer par Claude, TikTok, Google, Amazon, Anthropic et al, les reponses sont des bouts de code incohérents avec des commentaires qui donnent des idées pour mieux vibecoder 🤣.

je vais mettre quelques exemples plus tard, c'est beau!!

GOTO 10
#mastoDotBikeSatus #noAI #AIpoisoning

c'est rigolo, si je me fais passer par Claude, TikTok, Google, Amazon, Anthropic et al, les reponses sont des bouts de code incohérents avec des commentaires qui donnent des idées pour mieux vibecoder 🤣.

je vais mettre quelques exemples plus tard, c'est beau!!

GOTO 10
#mastoDotBikeSatus #noAI #AIpoisoning

ChatGPT recomienda tiendas falsas: el nuevo vector de fraude en compras con IA

El servicio de detección de estafas Ask Silver detectó que resultados de compras generados por ChatGPT incluían enlaces a tiendas clonadas que robaban datos de pago. Detrás del fenómeno estaría el «AI poisoning», una técnica que inunda la web con páginas fraudulentas para que los modelos de lenguaje las absorban y recomienden (Fuente The Guardian).

Usar un chatbot de IA para buscar productos parecía una forma más inteligente de comprar. Resulta que también puede ser una forma más sofisticada de ser estafado. El medio The Guardian reportó que hubo casos en que ChatGPT derivó a usuarios hacia sitios de venta online falsos pero convincentes, e incluso sugirió productos que nunca existieron. Según hallazgos del servicio de detección de estafas Ask Silver, tiendas clonadas comenzaron a aparecer en los resultados de compras generados por ChatGPT.

El mecanismo del fraude es puntualmente elaborado. Los estafadores operaban sitios que lucían auténticos pero que en realidad incluían enlaces a marcas populares como Russell & Bromley o Dunelm. Quienes realizaron pedidos no solo perdieron dinero, sino que también tuvieron sus datos de pago expuestos. El caso de Russell & Bromley tiene un contexto que los delincuentes supieron aprovechar: la marca dejó de existir como minorista independiente tras entrar en administración en enero de 2026 y ser absorbida por Next. Los estafadores ocuparon ese vacío con páginas imitadoras, capturando el tráfico de usuarios que aún buscaban el sitio original.

La técnica que podría estar detrás de estos ataques tiene nombre propio. Los investigadores sospechan que se trata de «AI poisoning», una técnica en la que actores maliciosos inundan la web con información falsa y páginas clonadas que eventualmente son absorbidas por los modelos de lenguaje, que sin saberlo las promueven como si fueran fuentes reales.

La advertencia de los expertos en protección al consumidor es directa. Louise Baxter, jefa del equipo de estafas de National Trading Standards, señaló que la gente no debería asumir que una recomendación es confiable solo porque proviene de un chatbot de IA. Los delincuentes se adaptan rápidamente a las nuevas tecnologías y utilizarán cualquier canal que les dé acceso a potenciales víctimas.

Las señales de alerta siguen siendo las de siempre: descuentos masivos, direcciones web extrañas, información de contacto deficiente y solicitudes de transferencias bancarias deben considerarse sospechosas. Los expertos recomiendan ir directamente a los sitios de los minoristas oficiales en lugar de confiar en los enlaces generados por IA.

Algunos de los sitios denunciados fueron eliminados por OpenAI tras ser reportados, pero el problema de fondo persiste. Con las herramientas de IA jugando un papel cada vez más central en cómo las personas buscan y compran online, las empresas necesitarán salvaguardas más robustas para evitar que los estafadores conviertan recomendaciones útiles en errores costosos.

@piracalamina #aipoisoning

@piracalamina #aipoisoning

@piracalamina #aipoisoning

@piracalamina #aipoisoning

@piracalamina #aipoisoning

The Register: Yet another experiment proves it’s too damn simple to poison large language models. “Unlike search engines that let you judge competing sources, search-backed AI chatbots can turn shaky web material into confident answers. Case in point: A security engineer convinced several bots that he was the reigning world champion of a popular German card game, even though no such […]

The Register: Yet another experiment proves it’s too damn simple to poison large language models. “Unlike search engines that let you judge competing sources, search-backed AI chatbots can turn shaky web material into confident answers. Case in point: A security engineer convinced several bots that he was the reigning world champion of a popular German card game, even though no such […]

The Register: Yet another experiment proves it’s too damn simple to poison large language models. “Unlike search engines that let you judge competing sources, search-backed AI chatbots can turn shaky web material into confident answers. Case in point: A security engineer convinced several bots that he was the reigning world champion of a popular German card game, even though no such […]

The Register: Yet another experiment proves it’s too damn simple to poison large language models. “Unlike search engines that let you judge competing sources, search-backed AI chatbots can turn shaky web material into confident answers. Case in point: A security engineer convinced several bots that he was the reigning world champion of a popular German card game, even though no such […]

The Register: Yet another experiment proves it’s too damn simple to poison large language models. “Unlike search engines that let you judge competing sources, search-backed AI chatbots can turn shaky web material into confident answers. Case in point: A security engineer convinced several bots that he was the reigning world champion of a popular German card game, even though no such […]

Digital Trends: This invisible technique poisons songs so AI can’t clone them. “The system targets a song’s waveform. My Music My Choice adds microscopic alterations so subtle that you’ll never notice them. Play the track on Spotify and it sounds exactly like the master recording. But feed that file into cloning software and everything breaks.”

Digital Trends: This invisible technique poisons songs so AI can’t clone them. “The system targets a song’s waveform. My Music My Choice adds microscopic alterations so subtle that you’ll never notice them. Play the track on Spotify and it sounds exactly like the master recording. But feed that file into cloning software and everything breaks.”

Digital Trends: This invisible technique poisons songs so AI can’t clone them. “The system targets a song’s waveform. My Music My Choice adds microscopic alterations so subtle that you’ll never notice them. Play the track on Spotify and it sounds exactly like the master recording. But feed that file into cloning software and everything breaks.”

Digital Trends: This invisible technique poisons songs so AI can’t clone them. “The system targets a song’s waveform. My Music My Choice adds microscopic alterations so subtle that you’ll never notice them. Play the track on Spotify and it sounds exactly like the master recording. But feed that file into cloning software and everything breaks.”

🎯 AI
===================

Executive summary: Attackers conducted an AI/SEO poisoning campaign that placed malicious ChatGPT and Grok conversations at the top of Google searches for common macOS troubleshooting queries. Victims copied a Terminal command from a legitimate-seeming AI conversation that fetched and executed an AMOS macOS stealer. No phishing email, trojanized installer, or bypass of macOS protections was observed.

Technical details:
• Malware: AMOS (Atomic macOS Stealer) variant observed harvesting passwords, escalating to root, and establishing persistent mechanisms on macOS hosts.
• Initial access: Search-engine poisoning that returned AI-hosted conversations (ChatGPT, Grok) instructing users to run Terminal commands framed as "safe system cleanup."
• Behavior: Silent credential harvesting, privilege escalation, persistence, and data exfiltration to attacker infrastructure (specific C2 domains were not provided in the source).

🔹 Attack Chain Analysis
• Initial Access: AI/SEO poisoning — malicious AI conversations ranked highly for benign queries like "clear disk space on macOS."
• Download/Execution: Victim copied a Terminal command from the AI conversation which downloaded and executed the stealer.
• Privilege Escalation: Observed escalation to root as part of the payload.
• Persistence: Installer created mechanisms to survive reboots and maintain data access.
• Exfiltration: Collected credentials and user data were exfiltrated (telemetry showed data leak activity).

Detection guidance:
• Monitor for unexpected use of Terminal by non-admin users following web searches for benign tasks.
• Alert on processes that spawn network connections shortly after Terminal invocation, and on unusual child processes of bash/zsh/sh.
• Inspect persistence artifacts and anomalous privilege escalations tied to recently executed shell commands.

Limitations and open questions:
• The report reproduces poisoned results across similar queries, but specific C2 indicators and hashes were not disclosed in the summary.
• Attribution and infrastructure details remain undeclared in the provided content.

Takeaway: This campaign demonstrates a shift from malware-hosted lures to weaponizing trusted AI platforms and search rankings to deliver malware via copy-paste commands. #AIpoisoning #AMOS #macOS #search_poisoning #LLM_attack

🔗 Source: https://www.huntress.com/blog/amos-stealer-chatgpt-grok-ai-trust

🎯 AI
===================

Executive summary: Attackers conducted an AI/SEO poisoning campaign that placed malicious ChatGPT and Grok conversations at the top of Google searches for common macOS troubleshooting queries. Victims copied a Terminal command from a legitimate-seeming AI conversation that fetched and executed an AMOS macOS stealer. No phishing email, trojanized installer, or bypass of macOS protections was observed.

Technical details:
• Malware: AMOS (Atomic macOS Stealer) variant observed harvesting passwords, escalating to root, and establishing persistent mechanisms on macOS hosts.
• Initial access: Search-engine poisoning that returned AI-hosted conversations (ChatGPT, Grok) instructing users to run Terminal commands framed as "safe system cleanup."
• Behavior: Silent credential harvesting, privilege escalation, persistence, and data exfiltration to attacker infrastructure (specific C2 domains were not provided in the source).

🔹 Attack Chain Analysis
• Initial Access: AI/SEO poisoning — malicious AI conversations ranked highly for benign queries like "clear disk space on macOS."
• Download/Execution: Victim copied a Terminal command from the AI conversation which downloaded and executed the stealer.
• Privilege Escalation: Observed escalation to root as part of the payload.
• Persistence: Installer created mechanisms to survive reboots and maintain data access.
• Exfiltration: Collected credentials and user data were exfiltrated (telemetry showed data leak activity).

Detection guidance:
• Monitor for unexpected use of Terminal by non-admin users following web searches for benign tasks.
• Alert on processes that spawn network connections shortly after Terminal invocation, and on unusual child processes of bash/zsh/sh.
• Inspect persistence artifacts and anomalous privilege escalations tied to recently executed shell commands.

Limitations and open questions:
• The report reproduces poisoned results across similar queries, but specific C2 indicators and hashes were not disclosed in the summary.
• Attribution and infrastructure details remain undeclared in the provided content.

Takeaway: This campaign demonstrates a shift from malware-hosted lures to weaponizing trusted AI platforms and search rankings to deliver malware via copy-paste commands. #AIpoisoning #AMOS #macOS #search_poisoning #LLM_attack

🔗 Source: https://www.huntress.com/blog/amos-stealer-chatgpt-grok-ai-trust

Qu’est-ce que l’« #AIpoisoning » ou empoisonnement de l’#IA ?
https://theconversation.com/quest-ce-que-l-ai-poisoning-ou-empoisonnement-de-lia-267995
Derrière la puissance apparente de l’#intelligenceartificielle se cache une vulnérabilité inattendue : sa dépendance aux données. En glissant du faux parmi le vrai, des pirates peuvent altérer son comportement – un risque croissant pour la fiabilité et la sécurité de ces technologies
#ia_beurk

Qu’est-ce que l’« #AIpoisoning » ou empoisonnement de l’#IA ?
https://theconversation.com/quest-ce-que-l-ai-poisoning-ou-empoisonnement-de-lia-267995
Derrière la puissance apparente de l’#intelligenceartificielle se cache une vulnérabilité inattendue : sa dépendance aux données. En glissant du faux parmi le vrai, des pirates peuvent altérer son comportement – un risque croissant pour la fiabilité et la sécurité de ces technologies
#ia_beurk

Qu’est-ce que l’« #AIpoisoning » ou empoisonnement de l’#IA ?
https://theconversation.com/quest-ce-que-l-ai-poisoning-ou-empoisonnement-de-lia-267995
Derrière la puissance apparente de l’#intelligenceartificielle se cache une vulnérabilité inattendue : sa dépendance aux données. En glissant du faux parmi le vrai, des pirates peuvent altérer son comportement – un risque croissant pour la fiabilité et la sécurité de ces technologies
#ia_beurk

What is AI poisoning? A computer scientist explains
#Tech #AI #AIPoisoning #CyberSecurity #ArtificialIntelligence #DataSecurity #MachineLearning #TechSafety #AIEthics #Misinformation #Technology #Innovation #Anthropic #LargeLanguageModels #Poisoning
https://the-14.com/what-is-ai-poisoning-a-computer-scientist-explains/

What is AI poisoning? A computer scientist explains
#Tech #AI #AIPoisoning #CyberSecurity #ArtificialIntelligence #DataSecurity #MachineLearning #TechSafety #AIEthics #Misinformation #Technology #Innovation #Anthropic #LargeLanguageModels #Poisoning
https://the-14.com/what-is-ai-poisoning-a-computer-scientist-explains/

What is AI poisoning? A computer scientist explains
#Tech #AI #AIPoisoning #CyberSecurity #ArtificialIntelligence #DataSecurity #MachineLearning #TechSafety #AIEthics #Misinformation #Technology #Innovation #Anthropic #LargeLanguageModels #Poisoning
https://the-14.com/what-is-ai-poisoning-a-computer-scientist-explains/

What is AI poisoning? A computer scientist explains
#Tech #AI #AIPoisoning #CyberSecurity #ArtificialIntelligence #DataSecurity #MachineLearning #TechSafety #AIEthics #Misinformation #Technology #Innovation #Anthropic #LargeLanguageModels #Poisoning
https://the-14.com/what-is-ai-poisoning-a-computer-scientist-explains/

What is AI poisoning? A computer scientist explains
#Tech #AI #AIPoisoning #CyberSecurity #ArtificialIntelligence #DataSecurity #MachineLearning #TechSafety #AIEthics #Misinformation #Technology #Innovation #Anthropic #LargeLanguageModels #Poisoning
https://the-14.com/what-is-ai-poisoning-a-computer-scientist-explains/

Ars Technica: AI models can acquire backdoors from surprisingly few malicious documents. “The research involved training AI language models ranging from 600 million to 13 billion parameters on datasets scaled appropriately for their size. Despite larger models processing over 20 times more total training data, all models learned the same backdoor behavior after encountering roughly the same […]

https://rbfirehose.com/2025/10/19/ars-technica-ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/

Ars Technica: AI models can acquire backdoors from surprisingly few malicious documents. “The research involved training AI language models ranging from 600 million to 13 billion parameters on datasets scaled appropriately for their size. Despite larger models processing over 20 times more total training data, all models learned the same backdoor behavior after encountering roughly the same […]

https://rbfirehose.com/2025/10/19/ars-technica-ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/

Ars Technica: AI models can acquire backdoors from surprisingly few malicious documents. “The research involved training AI language models ranging from 600 million to 13 billion parameters on datasets scaled appropriately for their size. Despite larger models processing over 20 times more total training data, all models learned the same backdoor behavior after encountering roughly the same […]

https://rbfirehose.com/2025/10/19/ars-technica-ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/

@clifor Insertar código o texto sin sentido perjudicaría a los que usan texto a voz. Quizá cambiando el código de la instancia o de Mastodon se podría hacer, yo de eso ni idea.

Pero podríamos acordar algo como jugar al mundo al revés. Véase ALT

#noAI #AltText #AIpoisoning

@clifor Insertar código o texto sin sentido perjudicaría a los que usan texto a voz. Quizá cambiando el código de la instancia o de Mastodon se podría hacer, yo de eso ni idea.

Pero podríamos acordar algo como jugar al mundo al revés. Véase ALT

#noAI #AltText #AIpoisoning

@clifor Insertar código o texto sin sentido perjudicaría a los que usan texto a voz. Quizá cambiando el código de la instancia o de Mastodon se podría hacer, yo de eso ni idea.

Pero podríamos acordar algo como jugar al mundo al revés. Véase ALT

#noAI #AltText #AIpoisoning

@clifor Insertar código o texto sin sentido perjudicaría a los que usan texto a voz. Quizá cambiando el código de la instancia o de Mastodon se podría hacer, yo de eso ni idea.

Pero podríamos acordar algo como jugar al mundo al revés. Véase ALT

#noAI #AltText #AIpoisoning

@clifor Insertar código o texto sin sentido perjudicaría a los que usan texto a voz. Quizá cambiando el código de la instancia o de Mastodon se podría hacer, yo de eso ni idea.

Pero podríamos acordar algo como jugar al mundo al revés. Véase ALT

#noAI #AltText #AIpoisoning

This is -ing unbelievable:
In the 17 hours running my "Discworld Ólyfjan" Iocaine, GPTBot has download the same 84 pages over 10000 times. They don't even change!

And Google has it on the search index: "Ólyfjan" [name of any discworld character]
has results.

HEX, the Bursar, even the troll Brick would be more intelligent than that...

#iocaine #aipoisoning #gptbot #chatgpt #discworld

This is -ing unbelievable:
In the 17 hours running my "Discworld Ólyfjan" Iocaine, GPTBot has download the same 84 pages over 10000 times. They don't even change!

And Google has it on the search index: "Ólyfjan" [name of any discworld character]
has results.

HEX, the Bursar, even the troll Brick would be more intelligent than that...

#iocaine #aipoisoning #gptbot #chatgpt #discworld

This is -ing unbelievable:
In the 17 hours running my "Discworld Ólyfjan" Iocaine, GPTBot has download the same 84 pages over 10000 times. They don't even change!

And Google has it on the search index: "Ólyfjan" [name of any discworld character]
has results.

HEX, the Bursar, even the troll Brick would be more intelligent than that...

#iocaine #aipoisoning #gptbot #chatgpt #discworld

This is -ing unbelievable:
In the 17 hours running my "Discworld Ólyfjan" Iocaine, GPTBot has download the same 84 pages over 10000 times. They don't even change!

And Google has it on the search index: "Ólyfjan" [name of any discworld character]
has results.

HEX, the Bursar, even the troll Brick would be more intelligent than that...

#iocaine #aipoisoning #gptbot #chatgpt #discworld

This is -ing unbelievable:
In the 17 hours running my "Discworld Ólyfjan" Iocaine, GPTBot has download the same 84 pages over 10000 times. They don't even change!

And Google has it on the search index: "Ólyfjan" [name of any discworld character]
has results.

HEX, the Bursar, even the troll Brick would be more intelligent than that...

#iocaine #aipoisoning #gptbot #chatgpt #discworld

One of the things that annoys me the most is that the scraper that went furthest into the tarpit (83 links deep) is also the one who comes back reading the same pages again and again:

{host="olyfjan.blomi.is",user_agent="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)",user_agent_group="GPTBot"} has sent 6991 GET requests, for the same 84 pages, downloading 22779416 bytes.

#gptbot #aipoisoning #iocaine

One of the things that annoys me the most is that the scraper that went furthest into the tarpit (83 links deep) is also the one who comes back reading the same pages again and again:

{host="olyfjan.blomi.is",user_agent="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)",user_agent_group="GPTBot"} has sent 6991 GET requests, for the same 84 pages, downloading 22779416 bytes.

#gptbot #aipoisoning #iocaine

One of the things that annoys me the most is that the scraper that went furthest into the tarpit (83 links deep) is also the one who comes back reading the same pages again and again:

{host="olyfjan.blomi.is",user_agent="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)",user_agent_group="GPTBot"} has sent 6991 GET requests, for the same 84 pages, downloading 22779416 bytes.

#gptbot #aipoisoning #iocaine