#aiinsoftwaredevelopment — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #aiinsoftwaredevelopment, aggregated by home.social.
-
Anthropic CEO Dario Amodei said AI will write 90% of the code for software engineers within three to six months, and every line within the year. More than a year later, this still hasn't happened.
Suppose he's eventually right about the volume. Follow what the volume does.
More code doesn't mean more value. Some changes introduce defects. Some features are poorly understood, so the tool built the wrong thing. More code increases the attack surface, and increases complexity, making it harder to add new features later.
More code appears mostly to equate to more Technical Debt. It takes from 2x to 10x as long to add new features as the code becomes a mess, and work in messy code is 15 times more likely to have defects (CodeScene: The Business Impact of Code Quality).
The Theory of Constraints tells us that if something isn't the bottleneck, don't optimize for it. Measure how long it takes to deliver value to the customer.
-
I asked for tests. I got tests, plus an implementation I never asked for.
For a recent CSD workshop, I expanded the Tennis Refactoring Kata to include scoring for sets, and gave IntelliJ Junie a tight prompt: build Jest tests for winning a set, and don't write the implementation code.
It did a credible job of generating the test cases themselves. They're simple, clear, and they test what I expected. There's not enough test cases, but that wasn't surprising. It also wrote the implementation code. That isn't what I wanted, but I've yet to find a way to avoid that.
I also tried the same prompt with qwen locally and got test cases that were more idiomatically correct, but it made fatal tennis mistakes (playerOne reaches 6 games before playerTwo has any, so the set is over at 6-0 even though the test case promises 6-2).
It continues to show that the current crop of GenAI tools can't generate code that I would want to rely on.
Try the same experiment on a problem you already know well.
-
I asked for tests. I got tests, plus an implementation I never asked for.
For a recent CSD workshop, I expanded the Tennis Refactoring Kata to include scoring for sets, and gave IntelliJ Junie a tight prompt: build Jest tests for winning a set, and don't write the implementation code.
It did a credible job of generating the test cases themselves. They're simple, clear, and they test what I expected. There's not enough test cases, but that wasn't surprising. It also wrote the implementation code. That isn't what I wanted, but I've yet to find a way to avoid that.
I also tried the same prompt with qwen locally and got test cases that were more idiomatically correct, but it made fatal tennis mistakes (playerOne reaches 6 games before playerTwo has any, so the set is over at 6-0 even though the test case promises 6-2).
It continues to show that the current crop of GenAI tools can't generate code that I would want to rely on.
Try the same experiment on a problem you already know well.
-
We use GenAI everyday, but have you ever stopped to think about how they work? These tools are Mimics. Mimicry isn't the same as intelligence, is it?
To demonstrate what these tools really do, I built a GenAI out of index cards. Five groups, three cards each, one die. A group is called on, they roll, the number picks the card, and their answer is added to the response. In a very simple fashion, we demonstrated how GenAI works. It isn't intelligent. It selects the next most-likely word (or, more accurately, 'token' - a partial word chunk) in the sequence.
Two things fell out of those ten minutes that people don't expect.
The first is that GenAI doesn't learn and doesn't have memory. Nothing from the first round is carried over to the next.
The second is about scale. ChatGPT/Claude/etc has about a trillion times as much data as our human-powered model. So it's not surprising that these tools are better mimics, because they have more parameters. However, they are still not intelligent.
Join me in rolling the die to start a conversation.