#ai-risk — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #ai-risk, aggregated by home.social.
-
AI Governance Is Racing Behind AI Adoption
https://youtu.be/v7XcaUeS1xY #AIGovernance #CyberSecurity #ArtificialIntelligence #AIRisk #GenAI #AgenticAI #InfoSec #DataPrivacy #CyberRisk #CISO #TechPolicy #AICompliance #DigitalTransformation #RiskManagement #AIAdoption -
Oh lord. Can we get a moment's peace? Anthropic's most powerful — and dangerous — AI tool has been compromised. A group on a private Discord gained unauthorized access to Claude Mythos, a cybersecurity model so capable it can exploit vulnerabilities faster than elite human hackers. They cracked it on launch day by guessing its URL. Access came via a third-party contractor. Anthropic says no core systems were breached, but the irony is hard to ignore: an AI built to defend against cyberattacks... got hacked. The group claims curiosity, not malice — but the risk is real. https://techcrunch.com/2026/04/21/unauthorized-group-has-gained-access-to-anthropics-exclusive-cyber-tool-mythos-report-claims
#Anthropic #ClaudeMythos #CyberSecurity #AIRisk #DataBreach #ProjectGlasswing #ArtificialIntelligence #TechNews #Hacked #AISecuriy -
Suspect Arrested For Allegedly Throwing Molotov Cocktail at Sam Altman's Home
-
Meta paused work with a $10B AI data vendor after hackers poisoned an open-source Python library called LiteLLM and walked out with four terabytes of data. So, that's bad. And the worst part? The stolen data might include the actual training methodologies that Meta, OpenAI, Anthropic, and Google paid billions to develop. Think about what that means. You can't protect your crown jewels if they're sitting inside a vendor who's connected to your three biggest competitors, all sharing the same open-source tools, all exposed by the same 40-minute window on PyPI before anyone noticed.
🎯 The attack chain here is worth understanding: hackers compromised a security scanner called Trivy, used that access to get credentials for a LiteLLM maintainer, then published two malicious package versions that lasted less than an hour before removal. Forty minutes. That's all it took.
💼 Mercor is not some sloppy startup. It's 22-year-old founders, $500M annualized revenue, and clients at the very top of the AI industry. Sophistication doesn't protect you from a poisoned dependency you never thought to audit.
🔍 The question I'd be asking right now if I were a CISO at any of these labs isn't "were we breached." It's "how many vendors in our training pipeline are running LiteLLM, and did we even know?"
Most companies audit their own software. Almost nobody audits the software their vendors use to build the data they're buying.
https://thenextweb.com/news/meta-mercor-breach-ai-training-secrets-risk
#Cybersecurity #AIRisk #SupplyChainSecurity spc #security #privacy #cloud #infosec #ThirdPartyRisk -
We keep worrying about AI doing something evil. Which it might, but right now, there’s a risk in the plumbing supporting it. Three vulnerabilities in LangChain and LangGraph, path traversal, unsafe deserialization, SQL injection. Not AI-specific attacks. They’re not novel nor sophisticated but these are the kinds of bugs we've been patching since the late '90s. One of them scored a severity of 9.3 out of 10. "The biggest threat to your enterprise AI data might not be as complex as you think." Remember that you're building AI on top of frameworks you didn't write, can't fully audit, and update whenever it's convenient. That's the actual problem.
🔐 Path traversal lets attackers read arbitrary files from the host system, including credentials
🔑 Unsafe deserialization exposes API keys and environment variables at runtime
🗄️ SQL injection in the checkpointing layer leaks conversation history from your AI agentsAll three are fixed now. But "fixed" only matters if you've actually applied the patches across every integration. Most organizations haven't.
The lesson isn't about AI security. It's that AI doesn't change what good security engineering looks like. Input validation, parameterized queries, strict path sandboxing. This is stuff your dev team learned before ChatGPT existed.
If you're deploying AI pipelines and you haven't done a security review of the frameworks underneath them, you're not running an AI strategy. You're running a trust exercise.
https://www.csoonline.com/article/4151814/langchain-path-traversal-bug-adds-to-input-validation-woes-in-ai-pipelines.html
#CyberSecurity #AIRisk #AppSec #security #privacy #cloud #infosec -
Forget sci-fi, the real AI risks are already here: cyberattacks, misinformation, and systems making life altering decisions with zero human oversight. 😳 Experts like Hinton & Bengio are warning us. Time to pay attention.
🔗 https://techglimmer.io/could-ai-actually-take-over-the-world/
#AI #ArtificialIntelligence #TechNews #AIRisk -
Two leading AI researchers wrote a book arguing that building superhuman AI will lead to human extinction. Their case: once AI surpasses us, there's no reliable way to control what it pursues.
Not everyone agrees. But the debate is worth following.
Here's the full story: https://www.pasadenastarnews.com/2026/03/28/everyone-dies-why-two-top-scientists-are-ai-doomers/
-
ContextHound v1.8.0 is out 🎉
This release adds a Runtime Guard API - a lightweight wrapper that inspects your LLM calls in-process, before the request hits OpenAI or Anthropic.
Free and open-source. If this is useful to you or your team, a GitHub star or a small donation helps keep development going.
github.com/IulianVOStrut/ContextHound#LLMSecurity #PromptInjection #CyberSecurity #OpenSource #AIRisk #AppSec #DevSecOps #GenAI #RuntimeSecurity #InfoSec #MLSecurity #ArtificialIntelligence
-
MODEL EXTRACTION: The AI Heist Nobody's Talking About
https://youtu.be/a6WtBx8l5Xk #CyberSecurity #AISecuity #MachineLearning #ModelStealing #ArtificialIntelligence #IPProtection #InfoSec #AIRisk #TechLeadership #DataProtection #ModelExtraction #CyberThreats #ResponsibleAI -
The biggest #AIrisk isn’t rogue agents, it’s silent failure at scale: As #AIsystems grow too complex for humans to fully understand or control, small errors can quietly compound over weeks. Despite most deployments still being early-stage, companies are racing to adopt AI out of fear of falling behind. Experts warn this #goldrushmentality leaves little room for #guardrails and the #consequences could tip the #economy into disorder. https://www.cnbc.com/2026/03/01/ai-artificial-intelligence-economy-business-risks.html?AIagents.at #AIagent #AI #LLM #GenAI
-
CW: Critical mention of llm, star trek tng, ai
I re-watched this episode of Star Trek recently. It is 38 years old. It's literally a warning against "vibe coding" and "prompt engineering" and not having boundaries on your AI.
https://en.wikipedia.org/wiki/Elementary%2C_Dear_Data
#startrek #tng #aislop #vibecoding #aibullshitmachines #ai #airisk
-
Data Poisoning — The Silent Sabotage of AI
https://youtu.be/J-tsemViDXk #Cybersecurity #ArtificialIntelligence #AIsecurity #DataPoisoning #MachineLearning #AIrisk #AISafety #ModelSecurity #FoundationModels #CyberRisk #Infosec #DigitalTrust -
New post: My key takeaways from Stuart Armstrong's "Smarter Than Us" — a 2014 book on AI risk that reads like it was written yesterday.
I read it in 2025 and merged Armstrong's warnings with what I've experienced using AI day-to-day. From AI agents to the moral philosophy problem, the gap between prediction and reality has narrowed dramatically.
https://www.ctnet.co.uk/key-takeaways-of-stuart-armstrong-smarter-than-us/
-
Prompt Injection Is the New Phishing. The most dangerous malware today doesn’t exploit code, it exploits instructions. https://youtu.be/Ze12t1iv81E #Cybersecurity #ArtificialIntelligence #AIsecurity #PromptInjection #AIGovernance #LLMSecurity #ThreatIntelligence #AIrisk #CISO
-
“The best way to predict the future is to invent it”*…
Dario Amodei, the CEO of AI purveyor Anthropic, has recently published a long (nearly 20,000 word) essay on the risks of artificial intelligence that he fears: Will AI become autonomous (and if so, to what ends)? Will AI be used for destructive pursposes (e.g., war or terrorism)? Will AI allow one or a small number of “actors” (corporations or states) to seize power? Will AI cause economic disruption (mass unemployment, radically-concentrated wealth, disruption in capital flows)? Will AI indirect effects (on our societies and individual lives) be destabilizing? (Perhaps tellingly, he doesn’t explore the prospect of an economic crash on the back of an AI bubble, should one burst– but that might be considered an “indirect effect,” as AI development would likely continue, but in fewer hands [consolidation] and on the heels of destabilizing financial turbulence.)
The essay is worth reading. At the same time, as Matt Levine suggests, we might wonder why pieces like this come not from AI nay-sayers, but from those rushing to build it…
… in fact there seems to be a surprisingly strong positive correlation between noisily worrying about AI and being good at building AI. Probably the three most famous AI worriers in the world are Sam Altman, Dario Amodei, and Elon Musk, who are also the chief executive officers of three of the biggest AI labs; they take time out from their busy schedules of warning about the risks of AI to raise money to build AI faster. And they seem to hire a lot of their best researchers from, you know, worrying-about-AI forums on the internet. You could have different models here too. “Worrying about AI demonstrates the curiosity and epistemic humility and care that make a good AI researcher,” maybe. Or “performatively worrying about AI is actually a perverse form of optimism about the power and imminence of AI, and we want those sorts of optimists.” I don’t know. It’s just a strange little empirical fact about modern workplace culture that I find delightful, though I suppose I’ll regret saying this when the robots enslave us.
Anyway if you run an AI lab and are trying to recruit the best researchers, you might promise them obvious perks like “the smartest colleagues” and “the most access to chips” and “$50 million,” but if you are creative you might promise the less obvious perks like “the most opportunities to raise red flags.” They love that…
– source
In any case, precaution and prudence in the pursuit of AI advances seems wise. But perhaps even more, Tim O’Reilly and Mike Loukides suggest, we’d profit from some disciplined foresight:
The market is betting that AI is an unprecedented technology breakthrough, valuing Sam Altman and Jensen Huang like demigods already astride the world. The slow progress of enterprise AI adoption from pilot to production, however, still suggests at least the possibility of a less earthshaking future. Which is right?
At O’Reilly, we don’t believe in predicting the future. But we do believe you can see signs of the future in the present. Every day, news items land, and if you read them with a kind of soft focus, they slowly add up. Trends are vectors with both a magnitude and a direction, and by watching a series of data points light up those vectors, you can see possible futures taking shape…
For AI in 2026 and beyond, we see two fundamentally different scenarios that have been competing for attention. Nearly every debate about AI, whether about jobs, about investment, about regulation, or about the shape of the economy to come, is really an argument about which of these scenarios is correct…
[Tim and Mike explore an “AGI is an economic singularity” scenario (see also here, here, and Amodei’s essay, linked above), then an “AI is a normal technology” future (see also here); they enumerate signs and indicators to track; then consider 10 “what if” questions in order to explore the implications of the scenarios, honing in one “robust” implications for each– answers that are smart whichever way the future breaks. They conclude…]
The future isn’t something that happens to us; it’s something we create. The most robust strategy of all is to stop asking “What will happen?” and start asking “What future do we want to build?”
As Alan Kay once said, “The best way to predict the future is to invent it.” Don’t wait for the AI future to happen to you. Do what you can to shape it. Build the future you want to live in…
Read in full– the essay is filled with deep insight. Taking the long view: “What If? AI in 2026 and Beyond,” from @timoreilly.bsky.social and @mikeloukides.hachyderm.io.ap.brid.gy.
[Image above: source]
* Alan Kay
###
As we pave our own paths, we might send world-changing birthday greetings to a man who personified Alan’s injunction, Doug Engelbart; he was born on this date in 1925. An engineer and inventor who was a computing and internet pioneer, Doug is best remembered for his seminal work on human-computer interface issues, and for “the Mother of All Demos” in 1968, at which he demonstrated for the first time the computer mouse, hypertext, networked computers, and the earliest versions of graphical user interfaces… that’s to say, computing as we know it, and all that computing enables.
https://youtu.be/B6rKUf9DWRI?si=nL09hD5GQD670AQO
#AI #AIRisk #artificalIntelligence #computerMouse #culture #DarioAmodei #DougEngelbart #graphicalUserInterfaces #history #hypertext #MikeLoukides #mouse #networkedComputers #scenarioPlanning #scenarios #Singularity #Technology #TimOReilly -
Anthropic CEO Dario Amodei released a 20,000-word essay warning that AI systems could cause millions of deaths within years - while simultaneously building those same systems. He describes a potential "country of geniuses in a datacenter" by 2027: 50 million minds smarter than Nobel laureates, running at 10x human speed. The contradiction between his warnings and his work highlights the industry's core tension.