First time I'm dealing with an incident where, at least in my opinion, the operator decided "AI take the wheel" and hand over post-exploitation to LLM. It would be funny, if there wasn't a real damage done. Nevertheless, few observations:
- LLM clearly has problem distinguishing operator's machine and the beachhead host. So, it leaked his working directory, with username and language when generating a script to copy files between hosts...
- Operator is pretending it is a pentest. So LLM documents everything. There are a lot of debug messages in logs. It checks
whoamiandhostnameway too often. - Post-exploitations scripts are very helpfully prefixed with a sequential number showing order of execution, making establishing timeline much easier.
- Everything is PowerShell except that one time, when it decided to rewrite
tasklistin C#... PowerShell constrained mode would have greatly helped, at least forcing it to be even more noisy. - It's not fast. I mean the execution is fast, but while the operator reviews output and writes new prompt it takes time, the whole intrusion still spans days and is super noisy.
- Everything is living-off-the-land and generic scripting. No malware.
- It has very uncanny feeling, just like everything AI. Looking at the logs and trying to follow some reasoning is difficult. Weird decisions that don't make sense.