#socalledai — Public Fediverse posts

Looking at this latest Microsoft AI assistant - "It manages your calendar, triages your inbox" - my eyebrow goes up and I say "does it now".

Forgotten already about the last person's wrongly-deleted emails? Got some new master-plan for why that can't happen this time?

Sounds to me like a triumph of optimism over experience.

https://www.404media.co/microsoft-wants-to-make-people-addicted-to-scout-its-new-ai-assistant-internal-documents-reveal/

#SoCalledAI #Microsoft

Looking at this latest Microsoft AI assistant - "It manages your calendar, triages your inbox" - my eyebrow goes up and I say "does it now".

Forgotten already about the last person's wrongly-deleted emails? Got some new master-plan for why that can't happen this time?

Sounds to me like a triumph of optimism over experience.

https://www.404media.co/microsoft-wants-to-make-people-addicted-to-scout-its-new-ai-assistant-internal-documents-reveal/

#SoCalledAI #Microsoft

Great post from Ky Decker. I especially appreciated this part, on the exhaustion of swimming against the tide:

"Do I ask my coworker to disable their note-taking tool, or do I allow them to record me? (Where does the data go? Who is reading it? Do we retain knowledge in the same way without manual note-taking?) Do I voice concerns over unread code entering the codebase, and the consequences of that pattern for institutional knowledge-building? Do I ask others on the design team to delay prototyping until later in the design process? Is it already too late to ask? Has the team already shipped the code, already designed the feature, already moved onto the next task? If someone requests my review on a pull request that was clearly vibe coded, do I review the code and write comments as usual, or send it back to them for self-review? Would initiating these discussions result in interpersonal stress? Should I just let things slide? Would I become known as a “difficult” coworker for pushing back on AI use? Does any of it really matter? Does anyone really care?

"All of these questions consumed energy. Whether I decided to confront them or not was moot: they left me tired and alienated either way."

https://ky.fyi/posts/ai-burnout

#SoCalledAI #LLM #coding #tech #burnout

Great post from Ky Decker. I especially appreciated this part, on the exhaustion of swimming against the tide:

"Do I ask my coworker to disable their note-taking tool, or do I allow them to record me? (Where does the data go? Who is reading it? Do we retain knowledge in the same way without manual note-taking?) Do I voice concerns over unread code entering the codebase, and the consequences of that pattern for institutional knowledge-building? Do I ask others on the design team to delay prototyping until later in the design process? Is it already too late to ask? Has the team already shipped the code, already designed the feature, already moved onto the next task? If someone requests my review on a pull request that was clearly vibe coded, do I review the code and write comments as usual, or send it back to them for self-review? Would initiating these discussions result in interpersonal stress? Should I just let things slide? Would I become known as a “difficult” coworker for pushing back on AI use? Does any of it really matter? Does anyone really care?

"All of these questions consumed energy. Whether I decided to confront them or not was moot: they left me tired and alienated either way."

https://ky.fyi/posts/ai-burnout

#SoCalledAI #LLM #coding #tech #burnout

Very interesting post about breaches and deletions involving LLM "agents". I feel like if I'd read this before yesterday's post, I'd have put the warning elements more strongly.

Here's two of the examples they mention which I thought were particularly illuminating.

1. This exploit actually happened the other day, affecting a Python package called LiteLLM:

"The malware searches the entire machine for private keys, AWS / GCP / Azure credentials, Kubernetes configs, database passwords, .gitconfig, crypto wallet files, etc and uploads them to the attacker’s server."

2. This second exploit is possible in principle if you give an LLM-bot access to your email program. "Although not seen in the wild yet, the mechanism is proven."

"An adversarial prompt embedded in an email is processed by an AI email assistant. The assistant generates a reply containing the same malicious prompt. The reply is sent. Recipients are infected without any human-to-human interaction."

If I understand correctly, this means that _any_ use of so-called "AI agents" puts at risk (for deletion, and potentially for stealing) everything to which that "agent" has access.

The thing is, you might _think_ you've told the bot what not to touch and what not to do, but that effectively means nothing. Once it's set going,

(a) it might accidentally _lose_ part of your original instruction (as in one of the other examples), or

(b) a malicious exploit might give it a _different_ instruction.

The only way to protect valuable data is to keep it separate from LLM "agents".

The writer's conclusion, which sounds correct to me:

"Isolation has to live outside of the agent’s context entirely. A built-in sandbox can be disabled by the agent (as Snowflake and Ona both demonstrated), whereas an OS-level containment presents a much more formidable obstacle since the agent has no direct mechanism to interact with it. As well, a properly sandboxed agent won’t have sensitive information (keys, etc) lying around for it to find, and won’t be able to connect to places that haven’t been allow-listed."

("Sandbox" in this context means an area where you can run software without it touching anything outside its boundaries.)

I think if I were gonna try this stuff out, I'd probably just do it on a separate machine, away from my real things. Any useful results could be transferred across later.

https://yoloai.dev/posts/ai-agent-threat-landscape/

#LLMs #SoCalledAI #AIAgents #security

Very interesting post about breaches and deletions involving LLM "agents". I feel like if I'd read this before yesterday's post, I'd have put the warning elements more strongly.

Here's two of the examples they mention which I thought were particularly illuminating.

1. This exploit actually happened the other day, affecting a Python package called LiteLLM:

"The malware searches the entire machine for private keys, AWS / GCP / Azure credentials, Kubernetes configs, database passwords, .gitconfig, crypto wallet files, etc and uploads them to the attacker’s server."

2. This second exploit is possible in principle if you give an LLM-bot access to your email program. "Although not seen in the wild yet, the mechanism is proven."

"An adversarial prompt embedded in an email is processed by an AI email assistant. The assistant generates a reply containing the same malicious prompt. The reply is sent. Recipients are infected without any human-to-human interaction."

If I understand correctly, this means that _any_ use of so-called "AI agents" puts at risk (for deletion, and potentially for stealing) everything to which that "agent" has access.

The thing is, you might _think_ you've told the bot what not to touch and what not to do, but that effectively means nothing. Once it's set going,

(a) it might accidentally _lose_ part of your original instruction (as in one of the other examples), or

(b) a malicious exploit might give it a _different_ instruction.

The only way to protect valuable data is to keep it separate from LLM "agents".

The writer's conclusion, which sounds correct to me:

"Isolation has to live outside of the agent’s context entirely. A built-in sandbox can be disabled by the agent (as Snowflake and Ona both demonstrated), whereas an OS-level containment presents a much more formidable obstacle since the agent has no direct mechanism to interact with it. As well, a properly sandboxed agent won’t have sensitive information (keys, etc) lying around for it to find, and won’t be able to connect to places that haven’t been allow-listed."

("Sandbox" in this context means an area where you can run software without it touching anything outside its boundaries.)

I think if I were gonna try this stuff out, I'd probably just do it on a separate machine, away from my real things. Any useful results could be transferred across later.

https://yoloai.dev/posts/ai-agent-threat-landscape/

#LLMs #SoCalledAI #AIAgents #security

Vexed with the Guardian for this poor-quality article on so-called AI.

"AI models that lie and cheat"

No. Maybe you mean that they did something which a human didn't want them to do - but a Large Language Model has no ability to conceive of truth or lies. What they do is to extrude statistically-likely text.

"deceptive scheming"

No. LLMs cannot "scheme".

"destroying emails and other files without permission"

Well, obviously they _did_ have permission - in the software sense - or they couldn't have done it.

A statistical word-order model isn't designed to follow instructions reliably. If you want to be sure that it can't delete files, then don't hook it up with file-deletion access.

(Or make separate backups first.)

"The research uncovered hundreds of examples of scheming."

Again, LLM-bots are not "scheming". They're just extruding text, based on probabilities calculated from older text.

"use cyber-attack tactics to reach their goals without being told they could do so."

This shows only that similar text sequences were in their training data already. Cyber-attack text in, cyber-attack text out. If you don't want your bot to actually _cause_ an attack, then don't pipe its output to channels where its unpredictable extrusions could have that effect.

"In one case unearthed in the CLTR research, an AI agent named Rathbun tried to shame its human controller who blocked them from taking a certain action. Rathbun wrote and published a blog accusing the user of “insecurity, plain and simple” and trying “to protect his little fiefdom”."

That part isn't even correct on its own terms! The blog seemingly by the Rathbun bot wasn't about "its human controller" - it was about a different person. (And hardly "unearthed" - that episode was slightly famous when it happened, and already much discussed.)

But also, "tried to shame" is projecting human motives onto a statistical model.

“The worry is that they’re slightly untrustworthy junior employees right now, but if in six to 12 months they become extremely capable senior employees scheming against you, it’s a different kind of concern.”

No. They're not "employees" and they're not "scheming". If humans fail to set appropriate technical limits on the scope of LLM-bot connections, that's the humans' fault.

And repeating anthropomorphic fantasies about them isn't helping! Fundamentally wrong framing. Pull your socks up, Guardian.

https://www.theguardian.com/technology/2026/mar/27/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says

#SoCalledAI #journalism

Vexed with the Guardian for this poor-quality article on so-called AI.

"AI models that lie and cheat"

No. Maybe you mean that they did something which a human didn't want them to do - but a Large Language Model has no ability to conceive of truth or lies. What they do is to extrude statistically-likely text.

"deceptive scheming"

No. LLMs cannot "scheme".

"destroying emails and other files without permission"

Well, obviously they _did_ have permission - in the software sense - or they couldn't have done it.

A statistical word-order model isn't designed to follow instructions reliably. If you want to be sure that it can't delete files, then don't hook it up with file-deletion access.

(Or make separate backups first.)

"The research uncovered hundreds of examples of scheming."

Again, LLM-bots are not "scheming". They're just extruding text, based on probabilities calculated from older text.

"use cyber-attack tactics to reach their goals without being told they could do so."

This shows only that similar text sequences were in their training data already. Cyber-attack text in, cyber-attack text out. If you don't want your bot to actually _cause_ an attack, then don't pipe its output to channels where its unpredictable extrusions could have that effect.

"In one case unearthed in the CLTR research, an AI agent named Rathbun tried to shame its human controller who blocked them from taking a certain action. Rathbun wrote and published a blog accusing the user of “insecurity, plain and simple” and trying “to protect his little fiefdom”."

That part isn't even correct on its own terms! The blog seemingly by the Rathbun bot wasn't about "its human controller" - it was about a different person. (And hardly "unearthed" - that episode was slightly famous when it happened, and already much discussed.)

But also, "tried to shame" is projecting human motives onto a statistical model.

“The worry is that they’re slightly untrustworthy junior employees right now, but if in six to 12 months they become extremely capable senior employees scheming against you, it’s a different kind of concern.”

No. They're not "employees" and they're not "scheming". If humans fail to set appropriate technical limits on the scope of LLM-bot connections, that's the humans' fault.

And repeating anthropomorphic fantasies about them isn't helping! Fundamentally wrong framing. Pull your socks up, Guardian.

https://www.theguardian.com/technology/2026/mar/27/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says

#SoCalledAI #journalism

"... the question almost nobody was asking, is not about Claude or any language model. It is a bureaucratic question about what happened to the kill chain, and the answer is Palantir. ...

"The target package for the Shajareh Tayyebeh school presented a military facility. ... This package looked like every other package in the queue. But outside the package, the school appeared in Iranian business listings. It was visible on Google Maps. A search engine could have found it. Nobody searched. At a thousand decisions an hour, nobody was going to. ...

"Someone decided to build a system that produces a thousand targeting decisions per hour and call them high-quality. Someone decided to start this war. Several hundred people are sitting on Capitol Hill, refusing to stop it. Calling it an “AI problem” gives those decisions, and those people, a place to hide."

- Kevin Baker, "Kill Chain"
https://artificialbureaucracy.substack.com/p/kill-chain

#war #death #tech #systems #narratives
#bureaucracy #SoCalledAI #Claude #Maven #Palantir

"... the question almost nobody was asking, is not about Claude or any language model. It is a bureaucratic question about what happened to the kill chain, and the answer is Palantir. ...

"The target package for the Shajareh Tayyebeh school presented a military facility. ... This package looked like every other package in the queue. But outside the package, the school appeared in Iranian business listings. It was visible on Google Maps. A search engine could have found it. Nobody searched. At a thousand decisions an hour, nobody was going to. ...

"Someone decided to build a system that produces a thousand targeting decisions per hour and call them high-quality. Someone decided to start this war. Several hundred people are sitting on Capitol Hill, refusing to stop it. Calling it an “AI problem” gives those decisions, and those people, a place to hide."

- Kevin Baker, "Kill Chain"
https://artificialbureaucracy.substack.com/p/kill-chain

#war #death #tech #systems #narratives
#bureaucracy #SoCalledAI #Claude #Maven #Palantir

The BBC, on identifying media as "AI-free":

"It is in response to fears that jobs or entire professions are being swept away in a wave of AI-powered automation."

Well you could also mention the issue of polluting the entire information landscape! That's a pretty important reason to want to know the provenance of things!

It isn't really "sweeping away" jobs, either. Not yet, anyway. More like, it's providing business owners with an excuse to cut staff and supply inferior services.

Shoddy framing overall in my opinion, BBC. You swallowed the hype.

https://www.bbc.co.uk/news/articles/cj0d6el50ppo

#SoCalledAI #BBC #journalism #media

The BBC, on identifying media as "AI-free":

"It is in response to fears that jobs or entire professions are being swept away in a wave of AI-powered automation."

Well you could also mention the issue of polluting the entire information landscape! That's a pretty important reason to want to know the provenance of things!

It isn't really "sweeping away" jobs, either. Not yet, anyway. More like, it's providing business owners with an excuse to cut staff and supply inferior services.

Shoddy framing overall in my opinion, BBC. You swallowed the hype.

https://www.bbc.co.uk/news/articles/cj0d6el50ppo

#SoCalledAI #BBC #journalism #media

About that stolen diagram with phrases like "continvouclous morging", a great little comment from WesolyKubeczek:

"I propose to adopt the word „morge”, a verb meaning „use an LLM to generate content that badly but recognizably plagiarizes some other known/famous work”.

"A noun describing such piece of slop could be „morgery”."

Morgery!

From comments at this Hacker News post:
https://news.ycombinator.com/item?id=47057829

Microsoft has now taken down the "morgery" but here is the original designer's explanation:
https://nvie.com/posts/15-years-later/

#Microsoft #morgery #SoCalledAI #language

About that stolen diagram with phrases like "continvouclous morging", a great little comment from WesolyKubeczek:

"I propose to adopt the word „morge”, a verb meaning „use an LLM to generate content that badly but recognizably plagiarizes some other known/famous work”.

"A noun describing such piece of slop could be „morgery”."

Morgery!

From comments at this Hacker News post:
https://news.ycombinator.com/item?id=47057829

Microsoft has now taken down the "morgery" but here is the original designer's explanation:
https://nvie.com/posts/15-years-later/

#Microsoft #morgery #SoCalledAI #language

@perigee

It's complicated :-)

I don't want to be reading/hearing LLM-generated stuff myself. I don't think it's a "healthy diet" for my knowledge of the world. When I engage with a communication, I'd like to know that a human was trying to tell me the truth as they understand it.

Or if I were coding something (not that I do tons of that), I'd want to go looking for what an expert geek says is a good way to implement this thing - preferably in a context where other people can comment "I wouldn't do it like that because XYZ" or "yes I agree this is the best way".

I don't approve of how generative LLMs & related tech are being deployed nonconsensually, nor the escalation in environmental degradation which they're part of, nor the exploitation of low-waged & traumatised moderators.

When I see other people using them...

If I respected the person in the first place, I'll probably have some curiosity about why and how.

That goes along with some doubt or apprehension about whether they know what they're getting into. For example, I think it's likely that LLM-generation prices will escalate drastically at some point, and anyone who's come to actually rely on it for their workflow (vs a bit of noodling to experiment) will get a shock.

And are they okay with the wider ethical picture, are they unaware, or are they thinking like "well everyone's doing it now, so it's not gonna make any difference if I do as well"?

I've actually been thinking recently that the social dynamic around LLMs reminds me of the social dynamic around masking. The average person doesn't know the full picture, but they also don't necessarily _want_ to know.

I've noticed that people distancing from so-called AI seem to be disproportionately (not always) the ones who know "how the sausage is made", with the less-techie people being more likely to marvel at it. That's a bit like how following covid science correlates with masking.

So like if you wanna play with LLMs, or if you wanna get on a bus or train unmasked... _I_ don't like the look of it. I'm probably gonna have a bit of a doubt that you've fully understood what the long-term consequences might be of that decision, for you & others. But I know I'm not the boss of you!

And I feel like the 6 years of covid awareness have sort of installed a boundary, made partly out of resignation and exhaustion, that I need to save my energy for helping people who _want_ to know.

#LLMs #SoCalledAI #covid

@perigee

It's complicated :-)

I don't want to be reading/hearing LLM-generated stuff myself. I don't think it's a "healthy diet" for my knowledge of the world. When I engage with a communication, I'd like to know that a human was trying to tell me the truth as they understand it.

Or if I were coding something (not that I do tons of that), I'd want to go looking for what an expert geek says is a good way to implement this thing - preferably in a context where other people can comment "I wouldn't do it like that because XYZ" or "yes I agree this is the best way".

I don't approve of how generative LLMs & related tech are being deployed nonconsensually, nor the escalation in environmental degradation which they're part of, nor the exploitation of low-waged & traumatised moderators.

When I see other people using them...

If I respected the person in the first place, I'll probably have some curiosity about why and how.

That goes along with some doubt or apprehension about whether they know what they're getting into. For example, I think it's likely that LLM-generation prices will escalate drastically at some point, and anyone who's come to actually rely on it for their workflow (vs a bit of noodling to experiment) will get a shock.

And are they okay with the wider ethical picture, are they unaware, or are they thinking like "well everyone's doing it now, so it's not gonna make any difference if I do as well"?

I've actually been thinking recently that the social dynamic around LLMs reminds me of the social dynamic around masking. The average person doesn't know the full picture, but they also don't necessarily _want_ to know.

I've noticed that people distancing from so-called AI seem to be disproportionately (not always) the ones who know "how the sausage is made", with the less-techie people being more likely to marvel at it. That's a bit like how following covid science correlates with masking.

So like if you wanna play with LLMs, or if you wanna get on a bus or train unmasked... _I_ don't like the look of it. I'm probably gonna have a bit of a doubt that you've fully understood what the long-term consequences might be of that decision, for you & others. But I know I'm not the boss of you!

And I feel like the 6 years of covid awareness have sort of installed a boundary, made partly out of resignation and exhaustion, that I need to save my energy for helping people who _want_ to know.

#LLMs #SoCalledAI #covid

@packetcat

In that post, the author describes the counted words as "thought or written", and that got me thinking about: is what an LLM does "writing"?

I'm leaning towards not - not in the sense that humans "write". I think for an LLM, it's more like "shuffling and extruding".

#LLMs #SoCalledAI

@packetcat

In that post, the author describes the counted words as "thought or written", and that got me thinking about: is what an LLM does "writing"?

I'm leaning towards not - not in the sense that humans "write". I think for an LLM, it's more like "shuffling and extruding".

#LLMs #SoCalledAI

“There may be moderators who escape psychological harm, but I’ve yet to see evidence of that... content moderation belongs in the category of dangerous work, comparable to any lethal industry.”

“People are hired under ambiguous labels, but only after contracts are signed and training begins do they realise what the actual work is.”

https://www.theguardian.com/global-development/2026/feb/05/in-the-end-you-feel-blank-indias-female-workers-watching-hours-of-abusive-content-to-train-ai

#work #trauma #data #moderation #SoCalledAI

“There may be moderators who escape psychological harm, but I’ve yet to see evidence of that... content moderation belongs in the category of dangerous work, comparable to any lethal industry.”

“People are hired under ambiguous labels, but only after contracts are signed and training begins do they realise what the actual work is.”

https://www.theguardian.com/global-development/2026/feb/05/in-the-end-you-feel-blank-indias-female-workers-watching-hours-of-abusive-content-to-train-ai

#work #trauma #data #moderation #SoCalledAI

@GossiTheDog

Woah, that is some sickening evasion of responsibility. How dare Musk & co pretend that a bot's apology is an adequate substitute for theirs.

Edited to add: I've since seen a thing saying the quasi-apology was in response to a Twitter user's prompt - as distinct from being directly stimulated by Musk & co. So Musk _could_ still be about to apologise, as the human in charge of the bot.

#Grok #Twitter #ElonMusk #ethics #LLMs #SoCalledAI

@GossiTheDog

Woah, that is some sickening evasion of responsibility. How dare Musk & co pretend that a bot's apology is an adequate substitute for theirs.

Edited to add: I've since seen a thing saying the quasi-apology was in response to a Twitter user's prompt - as distinct from being directly stimulated by Musk & co. So Musk _could_ still be about to apologise, as the human in charge of the bot.

#Grok #Twitter #ElonMusk #ethics #LLMs #SoCalledAI

@cy

Re "existing solutions taken away", "tiny little opt-out button" etc: this is a valid critique of the stats, and at the same time there _are_ people using LLMs unforced, maybe a lot of them.

A lot of what I've seen about coercion has been in the context of coding roles, and I think that makes sense. Techie people are aware of how it works and how illusory is the so-called "intelligence". They know "how the sausage is made". Plus, the risks of using LLMs for code are relatively obvious. So it does take coercion to make the sceptical techies go against their better judgement.

But a lot of people don't need coercing. They're like "so this is the new thing, let's try it out... ooh, this bit is going to help me".

- You can ask it for ideas and it'll give you some.

- You can ask it to summarise a meeting, and even when the summary isn't perfect, it's close enough to remind you what the meeting was about, and who's got actions.

Well, it's arguable that this cohort was manipulated or deceived or not given the full picture... but I don't think it makes sense to frame that as being _forced_. This is a different group from the "you'll be judged or punished for not using it" cohort. This group is genuinely curious, and even sometimes a bit enthusiastic about spreading the word among colleagues, "have you tried...?"

I think whether you've witnessed that experimental curiosity very much depends on the circles you move in, but it wouldn't totally shock me to discover it was quite common.

@zkat @mcc @xgranade @jplebreton @aud

#LLMs #SoCalledAI #stats

@cy

Re "existing solutions taken away", "tiny little opt-out button" etc: this is a valid critique of the stats, and at the same time there _are_ people using LLMs unforced, maybe a lot of them.

A lot of what I've seen about coercion has been in the context of coding roles, and I think that makes sense. Techie people are aware of how it works and how illusory is the so-called "intelligence". They know "how the sausage is made". Plus, the risks of using LLMs for code are relatively obvious. So it does take coercion to make the sceptical techies go against their better judgement.

But a lot of people don't need coercing. They're like "so this is the new thing, let's try it out... ooh, this bit is going to help me".

- You can ask it for ideas and it'll give you some.

- You can ask it to summarise a meeting, and even when the summary isn't perfect, it's close enough to remind you what the meeting was about, and who's got actions.

Well, it's arguable that this cohort was manipulated or deceived or not given the full picture... but I don't think it makes sense to frame that as being _forced_. This is a different group from the "you'll be judged or punished for not using it" cohort. This group is genuinely curious, and even sometimes a bit enthusiastic about spreading the word among colleagues, "have you tried...?"

I think whether you've witnessed that experimental curiosity very much depends on the circles you move in, but it wouldn't totally shock me to discover it was quite common.

@zkat @mcc @xgranade @jplebreton @aud

#LLMs #SoCalledAI #stats

Hadn't seen this variation before.

Recently, a colleague has had that thing where, un-asked-for by them, their "AI notetaker" tries to go to all the meetings in their calendar. This time it was double trouble: they somehow had _two_ bots trying to come into every meeting citing their name. Consequently, a few of us have had to boot out these bots at the start of a meeting, when the person themself wasn't there.

Yesterday, the _real_ person tried to go to a meeting. The host (who was a bit flustered due to connection problems) mistook their name for the bot appearing yet again, and kicked them out! so they missed the meeting!

(I found out only afterwards that they'd been trying to get on, and what had happened.)

=

Luckily it wasn't a meeting which hinged on this one person's presence, so not super high stakes. I was just thinking about it again now, and the phenomenon of "unwanted bot-behaviour causes knock-on problem". It reminded me of this more-troublesome episode, which I'd also read about yesterday:
https://www.osnews.com/story/143376/dark-patterns-killed-my-wifes-windows-11-installation/

#software #bots #SoCalledAI

Hadn't seen this variation before.

Recently, a colleague has had that thing where, un-asked-for by them, their "AI notetaker" tries to go to all the meetings in their calendar. This time it was double trouble: they somehow had _two_ bots trying to come into every meeting citing their name. Consequently, a few of us have had to boot out these bots at the start of a meeting, when the person themself wasn't there.

Yesterday, the _real_ person tried to go to a meeting. The host (who was a bit flustered due to connection problems) mistook their name for the bot appearing yet again, and kicked them out! so they missed the meeting!

(I found out only afterwards that they'd been trying to get on, and what had happened.)

=

Luckily it wasn't a meeting which hinged on this one person's presence, so not super high stakes. I was just thinking about it again now, and the phenomenon of "unwanted bot-behaviour causes knock-on problem". It reminded me of this more-troublesome episode, which I'd also read about yesterday:
https://www.osnews.com/story/143376/dark-patterns-killed-my-wifes-windows-11-installation/

#software #bots #SoCalledAI

Interesting explanation of LLM training frameworks and the incentives for confident guessing.

"The authors examined ten major AI benchmarks, including those used by Google, OpenAI and also the top leaderboards that rank AI models. This revealed that nine benchmarks use binary grading systems that award zero points for AIs expressing uncertainty.

" ... When an AI system says “I don’t know”, it receives the same score as giving completely wrong information. The optimal strategy under such evaluation becomes clear: always guess. ...

"More sophisticated approaches like active learning, where AI systems ask clarifying questions to reduce uncertainty, can improve accuracy but further multiply computational requirements. ...

"Users want systems that provide confident answers to any question. Evaluation benchmarks reward systems that guess rather than express uncertainty. Computational costs favour fast, overconfident responses over slow, uncertain ones."

=

My comment: "Fast, overconfident responses" sounds a bit similar to "bullshit", does it not?

https://theconversation.com/why-openais-solution-to-ai-hallucinations-would-kill-chatgpt-tomorrow-265107

#ChatGPT #LLMs #SoCalledAI

Interesting explanation of LLM training frameworks and the incentives for confident guessing.

"The authors examined ten major AI benchmarks, including those used by Google, OpenAI and also the top leaderboards that rank AI models. This revealed that nine benchmarks use binary grading systems that award zero points for AIs expressing uncertainty.

" ... When an AI system says “I don’t know”, it receives the same score as giving completely wrong information. The optimal strategy under such evaluation becomes clear: always guess. ...

"More sophisticated approaches like active learning, where AI systems ask clarifying questions to reduce uncertainty, can improve accuracy but further multiply computational requirements. ...

"Users want systems that provide confident answers to any question. Evaluation benchmarks reward systems that guess rather than express uncertainty. Computational costs favour fast, overconfident responses over slow, uncertain ones."

=

My comment: "Fast, overconfident responses" sounds a bit similar to "bullshit", does it not?

https://theconversation.com/why-openais-solution-to-ai-hallucinations-would-kill-chatgpt-tomorrow-265107

#ChatGPT #LLMs #SoCalledAI

"I dislike the attempt to gaslight people into swearing fealty to a sickly and frail psuedo-industry where everybody but NVIDIA and consultancies lose money."

https://www.wheresyoured.at/the-haters-gui/

(long)

#SoCalledAI #business #bubble #AIBubble #LLMs #NVidia

"I dislike the attempt to gaslight people into swearing fealty to a sickly and frail psuedo-industry where everybody but NVIDIA and consultancies lose money."

https://www.wheresyoured.at/the-haters-gui/

(long)

#SoCalledAI #business #bubble #AIBubble #LLMs #NVidia

Fascinating collection of firsthand experiences, gathered by Brian Merchant.

From a comment:

"I can’t help but notice that stories aren’t “I lost my job because AI is able to do it better”, they are “I lost my job because upper management is hype-pilling and thinks AGI is around the corner”. Which is a bad thing, but if we suppose for a moment that AGI is not around the corner, and AI is a bubble? Those jobs will be back with vengeance once technical debt catches up. ... when your codebase is now an AI-written mess without documentation and tests and diffused knowledge in heads of those who have written it, it will collapse sooner or later."
https://www.bloodinthemachine.com/p/how-ai-is-killing-jobs-in-the-tech-f39

#LLM #SoCalledAI #tech #jobs #coding #TechnicalDebt

Fascinating collection of firsthand experiences, gathered by Brian Merchant.

From a comment:

"I can’t help but notice that stories aren’t “I lost my job because AI is able to do it better”, they are “I lost my job because upper management is hype-pilling and thinks AGI is around the corner”. Which is a bad thing, but if we suppose for a moment that AGI is not around the corner, and AI is a bubble? Those jobs will be back with vengeance once technical debt catches up. ... when your codebase is now an AI-written mess without documentation and tests and diffused knowledge in heads of those who have written it, it will collapse sooner or later."
https://www.bloodinthemachine.com/p/how-ai-is-killing-jobs-in-the-tech-f39

#LLM #SoCalledAI #tech #jobs #coding #TechnicalDebt

Insightful thread from @skiles about the role of AI hype in channelling government money to millionaires & billionaires.

About the US, but I think relevant to the UK as well - remember Starmer's enthusiasm recently about so-called AI?

https://carhenge.club/@skiles/114203177462811812

#LLM #SoCalledAI #USPol #UKPol #Starmer

Insightful thread from @skiles about the role of AI hype in channelling government money to millionaires & billionaires.

About the US, but I think relevant to the UK as well - remember Starmer's enthusiasm recently about so-called AI?

https://carhenge.club/@skiles/114203177462811812

#LLM #SoCalledAI #USPol #UKPol #Starmer