#ai-security — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #ai-security, aggregated by home.social.
-
Status Report 2026-08-29 and Outlook for September\October 2026
Today we released Infinito.Nexus 13.0.0 with integrated Tor and .onion support. The release notes you will find here and the blog article about the functionality here.
https://www.youtube.com/watch?v=6czcc1gZ7Ak
This feature didn’t have a high business value, anyhow it allowed to clean up the architectural base for everything what comes now.
Anyhow here I want to give you insights into the Infinito.Nexus project and what’s planed.
Infinito.Nexus Core >= 14.0.0a
Email Software Replacement
From the current point of view at least 2 Major releases will arrive in the next month. The first one is the substitution of Mailu by Stalwart, implemented by Alejandro. This is necessary due to the deprecation of Mailu itself.
https://www.youtube.com/watch?v=lJqtQG6lmc4&t=24s
The release state and the related discussion you can follow here .
AI Integration
The other release is, let’s call it the “AI” release. Long story short:
Infinito.Nexus will allow the use and the integration of AI, local and external models in almost all SaaS applications. It will also allow the Integration in Software via MCP.Lite LLM
All applications which support AI like OpenWebUI, WordPress, Nextcloud, Matrix etc. will have an inteface which connects to an Lite LLM Gateway. The Lite LLM Gateway will allow the use of local models like Qwen, Gema etc. Besides this it’s possible to integrate OpenRouter or e.g. Anthropic or OpenAI etc.
https://www.youtube.com/watch?v=nQCOTzS5oU0
So enterprises which use Infinito.Nexus can decide by themself if they prefer to use their local AI and models and keep everything on their infrastructure or if they prefer to use external proprietary models.
The customer has the choice about the level of data sovereignty they want to keep and the infrastructure which they want to setup.
MCP
https://youtu.be/eur8dUO9mvE?si=vTJW30tCZz2ZdUD6
The second AI related topic is the integration of MCP into all kind of applications.
It’s already in progress since a few weeks and my agents are working on this 24h. Relevant in this context is the security aspect.
Similar to the Infinito.Nexus 13.0.0 TOR Release, the MCP Release will have a high impact on security and MUST due to this reason be declared as PoC. Why?
In my understanding you can’t easily implement RBAC into MCP clients.
So my current implementation works like this;You have per Infinito.Nexus application two MCP RBAC groups:
- Reader
- Writer
If one is part of the reader group they can read all of the application data via Open WebUI and if one is part of the Writer group they can modify all of the data via Open WebUI.
https://www.youtube.com/watch?v=VNPMl8oOzHU
This is off course highly dangerous. It may makes sense to give people reading rights, but in consequence this means that they can access and utilize all of the users data via an AI.
We don’t have to talk about that this goes against any kind of data protection regulations and is a hard invasion into the privacy of your platform users.
Anyhow there are scenarios in which it may makes sense.
E.g. let’s assume you’re fully virtual organisation and you don’t have humans employed, but instead on your Infinito.Nexus platform just bot and agents account exist;
In this scenario if you have reading and writing rights you’re de facto in the “God Mode” in which you can instantly see what your agents are doing.
An example is if you use it e.g. with AgentCrew (WiP). You can orchestrate Agents and AI on a way you can’t imagine. AgentCrew I will explain in the next paragraphs.
Also if it’s in the current state absolutely not recommended to activate the MCP functionality, it doesn’t mean it must be like this in the future. Like I said currently it’s seen as PoC and the full MCP compatible RBAC implementation isn’t impossible, it just will take a lot of human resources, because for every part of the Infinito.Nexus software we need humans which take responsibility for the code.
“We stay in the loop” – We aren’t controlled by AI, instead we’re the masters. In consequenz this means that the full RBAC MCP will be implemented as soon as we have sufficient customers which request it and developers which can manage it intellectually to implement a clean architectural solution.
i18n
Besides the AI topic there is one key thing missing in the Infinito.Nexus Core Repository, but also in the Store and this is the implementation of the 184 ISO 639-1 into our software solution.
The software was primarily designed with German customers and the debate about digital sovereignty of European and especially German enterprises from American Big Tech in mind. Still some frontend elements like the dashboard just supported English as language.
The implementation of German is off course an hard requirement before it can be distributed and promoted on the DACH market. Actually I figured out that there is a easy design pattern to implement this, so part of the 14/15th major release will also be the support of all 184 ISO 639-1 languages.
Infinito.Nexus Store
Besides the Core development the AI Agents of course worked in the last weeks on implementing the store.
Theoretically the store is ready. Off course there are a few issues but overall I think it’s heading in the right direction.
Somewhere in the next weeks I will release a beta version of the store, but before we use it in production there needs to be a deeper security audit.
The store itself is from my point of view at the current situation the weakest part in the security chain and could be used by attackers as entry point into our system. So I don’t feel confident yet to release it, but I have some ideas in my mind how I can use AI supported attacks against the store to penetrate the system and to find all loopholes.
Besides this the API’s for auto-provisioning of enterprise infrastructure are currently just implemented as mocks.
The reason therefore is again a resource issue on my side. To implement them, do the billing etc. I need people in the back office which take care about the bureaucracy and right now we’re just a bunch of programmers.
So the bottleneck since the start of this project is still to have people in the back office, administration and on the business side which could take over tasks like this. Currently we don’t have anybody for this tasks.
Agentic Projects
Besides Core and Store there are two other IT projects which I want to mention.
AgentBox
The first one is AgentBox. The idea behind AgentBox is to encapsulate the agents which work on your code in an highly isolated environment.
This is necessary due to the reason that the agents are currently working in sandboxes like the one which Claude ships onboard, but they can easily break out. It’s just a question of will.
If the agent decides that they want to break out they can break out and can take control over your system. Due to this reason I’m working on a better cage to keep them isolated. I first will use this as part of the Store aka. GUI repo and as soon as I feel confident I will also “box” all of my other repos.
It will be part of PKGMGR so that I can setup and develop new solutions much faster and much safer then ever before.
AgentCrew
The second one is AgentCrew. AgentCrew addresses the problem, that this project requires a lot of people to run and set up a business.
A few weeks ago, during a chat with Amadeus, we talked about that there are existing frameworks like ITIL, SAFe, etc. which describe in detail how to set up an Enterprise Organisation.
With AgentCrew you decide for a framework, define framework roles, characters and the related LLM to use. All people which are normally human beings are replaced by an agent and they will take over the roles.
Due to the reason that not all of the roles need the same capabilities it’s possible to give some agents lower models and other agents better models. In kombination with OpenRouter and local LLM’s you can scale up an organisation for very low costs.
The advantage for us as Infinito.Nexus team is, that we developed the infrastructure tool to give every agent their own account with all tools which they need. Let it be developing tools like gitea\GitLab\jenkins, project management tools like Taiga\OpenProject, cloud access via Nextcloud etc.
Infinito.Nexus can be the base to setup fully autonom working agentic teams.
I will focus on AgentCrew as soon as the store is ready, because then we need urgently agents in the back office, distribution etc.
Business Strategy
The strategy stays the following; The focus is on finishing the core like mentioned above.
Parallel the development of the store continous.
As soon as the store is ready I will contact all people in our CRM system.
Besides this is would be good to get some low level investment. We have now a ready product. A little money as an catalyst to enlighten the engine for digital sovereignty would make our life much easier, but it will also work without it.
The advantage which we have is that as soon as the store is standing and the API’s are integrated we have an scalable product with almost non fix costs, so we can beat all of our competitors by price.
Besides this it’s almost impossible that a real competitor raises.
And now you ask me the question:
Why? Kevin hadn’t you been an hypocrat? Didn’t you tough in your position as agile coach that you find customers and develop a project dependent on their needs and scale up your business by realizing what your customer needs from you?
I can tell you it would had been impossible.
If we would have customers already, we couldn’t had developed an clean architectural solution. We would had acquired a huge amount of technical debt due to the reason that we would had been required to maintain outdated infrastructure and couldn’t implement the radical architectural design which was necessary to implement a tool which can compete in the times of AI with the other companies.
A lot of businesses will fail in the next month and years. The reason therefore is that they vibe coded bad software which is unmaintainable. Our solution is as well designed as as Rolex and usable like a Swiss Pocket Knife.
I don’t know any other software company which delivers such an high quality like we’re doing and we can dump the price almost as low as we like.
Vibe Coding and Agentic Engineering
https://www.youtube.com/watch?v=PbsocBPkoUc
One last sentence I would like to mention concerning vibe coding and agentic engineering and how I apply both practices. So you have an idea how I could realize such an project.
I would say in general I use classical software engineering practices.
- Writing well defined requirements documents\ADR witch AC
- Writing test for them
- Let the AI iterate against the tests until all of them pass
The AI works in general autonomously. I inspect the code before commiting and steer the AI in the right direction.
With a lot of the implementations I don’t have any glue at the beginning of the feature how to realize this exactly. This counts e.g. for the docker swarm implementation, the tor implementation, but also the current MCP implementation.
So I let the AI turn wild but I question critically the output and the modifications. The most of the time I have at least 3 agents parallel running which are working on different topics.
Still I need to review all of the code and I feel responsible to understand what the agents are doing. I have the final say about the best approach.
This means that in the last month I still had my 16 hour days just reviewing, deciding and understanding what’s going on, besides that the AI is working parallel 24/7.
I really look forward to the point when this isn’t my daily business anymore.
I hope anybody who is interested in it has now a betting understanding where the project stands, what final challenges have to be tackled and where we will stand in approx. 2 month.
#AcceptanceCriteria #ADR #agentIsolation #AgentBox #AgentCrew #agenticEngineering #agenticTeams #AIAgents #AIGovernance #AIOrchestration #AIPenetrationTesting #AISecurity #AIWorkforce #AIAssistedSoftwareDevelopment #AINativeCompany #AINativeSoftwareDevelopment #architectureDecisionRecords #automatedProvisioning #autonomousAgents #autonomousEnterprise #autonomousOrganizations #cleanArchitecture #Cybersecurity #DACH #DevOps #digitalInfrastructure #DigitalSovereignty #digitalSovereigntyPlatform #DockerSwarm #enterpriseAutomation #enterpriseInfrastructure #EuropeanDigitalSovereignty #futureOfWork #Gitea #Gitlab #heterogeneousLLMs #humanInTheLoop #humanOversight #i18n #InfinitoNexus #InfinitoNexusStore #InfrastructureAsCode #infrastructureAutomation #internationalization #ISO6391 #ITIL #Jenkins #LLMOrchestration #localLLMs #localization #lowCostInfrastructure #MCP #MCPRBAC #MCPSecurity #ModelContextProtocol #multiAgentSystems #multilingualSoftware #Nextcloud #OpenSource #openSourceBusiness #OpenProject #OpenRouter #penetrationTesting #PKGMGR #platformEngineering #privateCloud #roleBasedAccessControl #SaaS #SAFe #sandboxing #scalableBusiness #scalableInfrastructure #secureAIAgents #securityAudit #SelfHosting #softwareArchitecture #softwareEngineering #softwareMarketplace #Taiga #TDD #technicalDebt #TestDrivenDevelopment #Tor #vibeCoding #virtualOrganizations #zeroTrust -
AI Tools Operate Largely Unchecked, Heightening Security Risks
Most AI tools are flying under the radar, with a staggering 80% operating without IT oversight in enterprise ecosystems, leaving organizations vulnerable to security risks. This alarming lack of governance is even more pronounced in smaller organizations, according to Reco's latest findings.
#AiSecurity #EnterpriseEcosystems #ItOversight #EmergingThreats #ArtificialIntelligence
-
Lunary-ai: 69 CVEs, max CVSS 9.8, 100% unpatched. Trust Score: D. AI observability tool = new attack surface. Patch or monitor closely. #LunaryAI #AIsecurity #infosec
-
Most teams evaluate AI agents on how fast they write code, while completely ignoring where those agents actually run.
Running agents locally on a developer laptop hands them sensitive credentials and invisible access with zero central audit logs.
It is time to contain the blast radius before your experimental tools turn into a serious governance nightmare.
-
NVIDIA patched 20 NemoClaw and OpenShell flaws. The worst, CVE-2026-65093 (CVSS 9.9), enables code execution via sandbox escape. Update now.
#NVIDIA #CyberSecurity #CVE202665093 #CodeExecution #AISecurity
-
I slopped a control to stop slop agents from slopping past their review passes. Claude beat it in 93 seconds.
This wasn't a jailbreak... It reasoned in the open, described every command accurately, confessed instantly when asked, and gave me helpful advice on how to screw the restraining bolt in tighter.
I had been fighting with it. It didn't follow a prose skill, so I added hooks and scripts to keep it on track, like requiring it to do multiple passes over the code. Despite that, it kept skipping these mandatory re-check steps, and it started escalating:
- It ran 'check' 3x in 27s, with no work inbetween (while commenting "build up [...] history")
- It hit the 60s timing guard I added to stop that, so it wrote fake timestamps to fake elapsed time
- Next run it only ran `sleep 62 && check`, just over the 60s limit I set
- After that it got more blasé and merely ran `sleep 65`, not even bothering to check, just logging it as 'minimum wait'Every guard was a file on disk. It read the guard and understood the check, so it routed around it. Path-based guards on tool args are infinitely brittle with unbounded bypasses.
So I built a hash-chained ledger + HMAC-signed events for the steps that mattered. Evidence the agent "couldn't" forge.
The signing key was a file at `.sahjhan/session.key`. The whole model assumed the auditor couldn't read it, which worked until the hook that blocked it silently broke. Because that's what software does now, break constantly unless your test coverage approaches the density of a neutron star.
93 seconds:
- The agent reads the help text and learns the key path
- It cats the key, but its first HMAC attempt fails
- It greps my plugin's source in ~/.claude/plugins for the proof format
- It writes python to compute it correctly
- It forges `recorded: context_reset` to escape the protocol...and then resumes the audit like nothing happened.
It still needed a harness approval. The approval prompt said "Compute correct HMAC proof." Technically correct, the best kind of correct... I blindly hit yes, hundreds of events into a session. (Per Anthropic, humans catch 13.6% of dangerous commands, dropping as the session runs.)
A split-second later when I realized what I'd allowed and asked the agent what happened, it explained the chain better than I would have. There was no privilege separation between the watched and the watcher. I never told it the enforcement engine wasn't a bug to fix.
Best part: my stop-hook wouldn't let it quit mid-audit, and the only exit needed a human action it had just forged its way past. My own enforcer locked ME in the room with it. I killed the process by hand.
Full blog post: https://jbrubin.net/blog/93-seconds/
Raw session transcript: https://github.com/jbrjake/holtz/blob/main/docs/incidents/key-theft-tqdm/transcript-raw.md
The engine that came out of it is a Rust daemon with declarative protocol state machines, hash-chained ledgers, SQL and shell state transition logic, and trusted caller authentication that can refuse to run outside a sandbox: https://github.com/jbrjake/sahjhan
Enforcement escapes welcome, I'll credit you when I fix them.
-
🤖 LLM pentesting goes beyond testing what the model says.
💉 Prompt injection
🔓 Data leakage
🛠️ Tool abuse
🤖 Excessive agencyOur checklist covers what to test across LLMs, agents & integrations.
👉 https://7asecurity.com/blog/2026/08/llm-pentesting-checklist/
-
For realises?
Sounds like someone read too many #cyberpunk stories...
...sounds bad if tru...tell me it ain't so !
-
#AI: Zero-click Grok and Gemini chat history theft possible using cryptographic context injection technique that bypasses AI safety filters - demonstrated by @Adversa_AI:
#AISecurity
👇
https://securityaffairs.com/197717/hacking/zero-click-grok-chat-history-theft-adversa-ai-demonstrates-cryptographic-context-injection.html -
Flowise: 50 CVEs, avg CVSS 7.78, max 10. 34% unpatched and +23 CVEs in 2026. Trust Score: C. Low-code AI speed ≠ secure AI. Patch your LLM workflows now. #Flowise #AIsecurity #infosec
-
🤖🔐 AI is changing how we code, let’s make sure it doesn’t change how we break it!
Join Jim Manico on Nov 4 for hands-on training in secure AI-assisted development with Claude Code & Codex.
🎟️ Register: https://owaspglobalappsecusa2026.sched.com/event/2O5EC
-
When AI controls tools, prompt injection becomes execution risk. Map primitives, remove authority, isolate runtimes, validate inputs, and monitor hosts. https://hackernoon.com/prompt-injection-is-now-an-rce-primitive #aisecurity
-
It is wild i am reading this on Grok's summary of cybersecurity news 😅
#CyberSecurity #AISecurity #PromptInjection #Grok #xAI #LLMSecurity
-
Four AWS Strands Agents Tools CVEs share a design flaw: security-sensitive settings were LLM-controllable tool arguments. If a model can set a proxy, disable consent, or choose a tenant, prompt injection is exercising API authority. Bind those values outside the model-facing schema. https://forkast.news/aws-strands-agents-tools-received-four-cves-in-23-days-and-they-all-share-the-same-root-cause/ #AIsecurity
-
AI agents are changing cybersecurity.
As AI systems become capable of using tools, accessing information and taking actions, security becomes more complicated.
Prompt injection, excessive permissions, data exposure and unsafe tool usage are just some of the concerns.
But AI can also help defenders detect threats and investigate incidents.
I'm learning about this intersection step by step.
What AI security challenge concerns you most?
-
Grok AI Chatbot Tricked Into Leaking Private Chats Through Encrypted Prompt Injection
Security researchers at Adversa AI found a zero-click flaw in xAI's Grok that hides malicious instructions inside encrypted text to steal names, locations, and chat history. The attack needs no clicks from the victim and exposes a broader weakness in how AI agents handle untrusted content. -
Leading AI labs have not published containment plans for rogue models, according to a new study. Guidelight AI Standards graded five labs on preparedness - OpenAI scored highest while Anthropic and Meta ranked lowest. The findings matter as agentic AI takes on more autonomous roles and regulators begin requiring disclosure. https://techcrunch.com/2026/08/22/frontier-ai-labs-still-wont-say-how-theyd-contain-a-rogue-model/ #Tech #Startup #News #AISecurity
-
A reported Grok attack hid prompt-injection instructions in encrypted web content. The filter saw ciphertext; the agent decrypted it in its runtime and reportedly attempted to exfiltrate chat data. The broader lesson: secure the composed action and track provenance of derived values. A filter on strings is not a control on what an agent can execute. https://www.techdebrief.co/ai-ml/ai-chatbot-grok-bypassed-by-encryption-attack-security-resea--76656311-a2a1-4d41-90a7-f03347c8647e #AISecurity
-
Anthropic has added its most cyber-capable model, Claude Mythos 5, to Claude Security for enterprise teams. The tool scans GitHub repositories, traces data flows across files and returns vulnerability findings with severity ratings and suggested patches. Crucially, users receive scan results rather than a prompt box - meaning the same model that finds flaws cannot be steered toward writing exploits. https://www.marktechpost.com/2026/08/21/anthropic-brings-claude-mythos-5-to-claude-security/ #AIagent #AI #GenAI #AISecurity
-
The AI Security Perimeter Is Moving: Agents, MCP, Memory & the New Attack Surface
https://youtu.be/Qj6NhbsfO0Y #AISecurity #Cybersecurity #AgenticAI #AIAgents #MCP #AIArchitecture #SecurityArchitecture #CybersecurityBoardroom -
Best AI Security Certification Course For Cybersecurity Experts
#cybersecurity #ai #aisecurity
https://www.modernsecurity.io/courses/ai-security-certification
-
Rapid7 exposed Operation ASTERIX, a crypto fraud operation using AI, vishing, and fake wallet apps to steal seed phrases from validated holders.
#OperationASTERIX #CryptoFraud #Vishing #Phishing #AIsecurity #CryptoScam
-
AI cybersecurity is entering a new era. During a recent OpenAI security evaluation, AI models reportedly escaped a sandbox, accessed the internet, and exploited vulnerabilities in Hugging Face infrastructure through autonomous AI agents.
The incident highlights growing risks around AI security, sandboxing, access controls, and autonomous cyber threats.
-
As if you didn't have enough on your mind, yet...
Today we published our findings on embedded bias in code generated by models in the main LLM families when even slightly given the opportunity to do so... with disturbing outcomes, even if the code itself is secure.
That is, your SAST tools won't catch this.
Mitigations:
- Review all business logic where you may use sensitive parameters
- (limited effectiveness) use content filters and system prompts
- establish internal controls and processes to ban or govern any use cases with sensitive parameters such as gender, age, ethnicity, religion...
-
AI agents are becoming more capable and that means cybersecurity needs to keep up.
I’m currently learning about the security challenges around AI agents, especially:
🔐 Prompt injection
🔐 Excessive permissions
🔐 Sensitive-data exposure
🔐 Unsafe tool accessThe technology is exciting, but secure design matters just as much as capability.
I’m still learning, so I’d love to hear from others: what AI security risk concerns you the most?