#empiricism — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #empiricism, aggregated by home.social.
-
I saw this on Mastodon and almost had a stroke.
@davidgerard wrote:
“Most of the AI coding claims are conveniently nondisprovable. What studies there are show it not helping coding at all, or making it worse
But SO MANY LOUD ANECDOTES! Trust me my friend, I am the most efficient coder in the land now. No, you can’t see it. No, I didn’t measure. But if you don’t believe me, you are clearly a fool.
These guys had one good experience with the bot, they got one-shotted, and now if you say “perhaps the bot is not all that” they act like you’re trying to take their cocaine away.”
First, the term is falsifiable, and proving propositions about algorithms (i.e., code) is part of what I do for a living. Mathematically human-written code and AI-written code can be tested, which means you can falsify propositions about them. You would test them the same way.
There is no intrinsic mathematical distinction between code written by a person and code produced by an AI system. In both cases, the result is a formal program made of logic and structure. In principle, the same testing techniques can be applied to each. If it were really nondisprovable, you could not test to see what is generated by a human and what is generated by AI. But you can test it. Studies have found that AI-generated code tends to exhibit a higher frequency of certain types of defects. So, reviewers and testers know what logic flaws and security weaknesses to look for. This would not be the case if it were nondisprovable.
You can study this from datasets where the source of the code is known. You can use open-source pull requests identified as AI-assisted versus those written without such tools. You then evaluate both groups using the same industry-standard analysis tools: static analyzers, complexity metrics, security scanners, and defect classification systems. These tools flag bugs, vulnerabilities, performance issues, and maintainability concerns. They do so in a consistent way across samples.
A widely cited analysis of 470 real pull requests reported that AI-generated contributions contained roughly 1.7 times as many issues on average as human-written ones. The difference included a higher number of critical and major defects. It also included more logic and security-related problems. Because these findings rely on standard measurement tools — counting defects, grading severity, and comparing issue rates — the results are grounded in observable data. Again, I am making a point here. It’s testable and therefore disproveable.
This is a good paper that goes into it:
In this paper, we present a large-scale comparison of code authored by human developers and three state-of-the-art LLMs, i.e., ChatGPT, DeepSeek-Coder, and Qwen-Coder, on multiple dimensions of software quality: code defects, security vulnerabilities, and structural complexity. Our evaluation spans over 500k code samples in two widely used languages, Python and Java, classifying defects via Orthogonal Defect Classification and security vulnerabilities using the Common Weakness Enumeration. We find that AI-generated code is generally simpler and more repetitive, yet more prone to unused constructs and hardcoded debugging, while human-written code exhibits greater structural complexity and a higher concentration of maintainability issues. Notably, AI-generated code also contains more high-risk security vulnerabilities. These findings highlight the distinct defect profiles of AI- and human-authored code and underscore the need for specialized quality assurance practices in AI-assisted programming.
https://arxiv.org/abs/2508.21634
Something I’ve started to notice about a lot of the content on social media platforms is that most of the posts people are liking, sharing, and memetically mutating—and then spreading virally—usually don’t include any citations, sources, or receipts. It’s often just some out-of-context screenshot with no reference link or actual sources.
A lot of the anti-AI content is not genuine critique. It’s often misinformation, but people who hate AI don’t question it or ask for sources because it aligns with their biases. The propaganda on social media has gotten so bad that anything other than heavily curated and vetted feeds is pretty much useless, and it’s filled with all sorts of memetic contagions with nasty hooks that are optimized for you algorithmically. I am at the point where I will disregard anything that is not followed up with a source. Period. It is all optimized to persuade, coerce, or piss you off. I am only writing about this because this I’m actually able to contribute genuine information about the topic.
That they said symbolic propositions written by AI agents (i.e., code) are non-disprovable because they were written by AI boggles my mind. It’s like saying that an article written in English by AI is not English because AI generated it. It might be a bad piece of text, but it’s syntactically, semantically, and grammatically English.
Basically, any string of data can be represented in a base-2 system, where it can be interpreted as bits (0s and 1s). Those bits can be used as the basis for symbolic reasoning. In formal propositional logic, a proposition is a sequence of symbols constructed according to strict syntax rules (atomic variables plus logical connectives). Under a given semantics, it is assigned exactly one truth value (true or false) in a two-valued logic system.
They are essentially saying that code written by AI is not binary, isn’t symbolically logical at all, and cannot be evaluated as true or false by implying it is nondisproveable. At the lowest level, compiled code consists of binary machine instructions that a processor executes. At higher levels, source code is written in symbolic syntax that humans and tools use to express logic and structure. You can also translate parts of code into formal logic expressions. For example, conditions and assertions in a program can be modeled as Boolean formulas. Tools like SAT/SMT solvers or symbolic execution engines check those formulas for satisfiability or correctness. It blows my mind how confidently people talk about things they do not understand.
Furthermore that they don’t realize the projection is wild to me.
@davidgerard wrote:
“But SO MANY LOUD ANECDOTES! Trust me my friend, I am the most efficient coder in the land now. No, you can’t see it. No, I didn’t measure. But if you don’t believe me, you are clearly a fool.”
They are presenting a story—i.e., saying that the studies are not disprovable—and accusing computer scientists of using anecdotal evidence without actually providing evidence to support this, while expecting people to take it prima facie. You’re doing what you are accusing others of doing.
It comes down to this: they feel that people ought not to use AI, so they are tacitly committed to a future in which people do not use AI. For example, a major argument against AI is the damage it is doing to resources, which is driving up the prices of computer components, as well as the ecological harm it causes. They feel justified in lying and misinforming others if it achieves the outcome they want—people not using AI because it is bad for the environment. That is a very strong point, but most people don’t care about that, which is why they lie about things people would care about.It’s corrupt. And what’s really scary is that people don’t recognize when they are part of corruption or a corrupt conspiracy to misinform. Well, they recognize it when they see the other side doing it, that is. No one is more dangerous than people who feel righteous in what they are doing.
-
In the Way of Inquiry • Reconciling Accounts
• https://inquiryintoinquiry.com/2023/01/24/in-the-way-of-inquiry-reconciling-accounts-a/The Reader may share with the Author a feeling of discontent at this point, attempting to reconcile the formal intentions of this inquiry with the cardinal contentions of experience. Let me try to express the difficulty in the form of a question:
What is the bond between form and content in experience, between the abstract formal categories and the concrete material contents residing in experience?
Once toward the end of my undergrad years a professor asked me how I'd personally define mathematics and I told him I saw it as “the form of experience and the experience of form”. This is not the place to argue for the virtues of that formulation but it does afford me one of the handles I have on the bond between form and content in experience.
I have no more than a tentative way of approaching the question. I take there to be a primitive category of “form‑in‑experience” — I don’t have a handy name for it yet but it looks to have a flexible nature which from the standpoint of a given agent easily passes from the “structure of experience” to the “experience of structure”.
Overview
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_OverviewObstacles
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_Part_5#Obstacles#Peirce #Inquiry #InquiryIntoInquiry #InquiryDrivenSystems
#Semiotics #SignRelations #Semiositis #ObstaclesToInquiry
#Logic #Abduction #Deduction #Induction #ScientificMethod
#Experience #Expectation #EffectiveDescription #FiniteMeans
#Abstraction #Analogy #Form #Matter #Empiricism #Rationalism
#Concretion #Information #Comprehension #Extension #Intension -
In the Way of Inquiry • Material Exigency 2
• https://inquiryintoinquiry.com/2023/01/20/in-the-way-of-inquiry-material-exigency-a/A turn of events so persistent must have a cause, a force of reason to explain the dynamics of its recurring moment in the history of ideas. The nub of it's not born on the sleeve of its first and last stages, where the initial explosion and the final collapse march along their stubborn course in lockstep fashion, but is embodied more naturally in the middle of the above narrative.
Experience exposes and explodes expectations. How can experiences impact expectations unless the two types of entities are both reflected in one medium, for instance and perhaps without loss of generality, in the form of representation constituting the domain of signs?
However complex its world may be, internal or external to itself or on the boundaries of its being, a finite creature's description of it rests in a finite number of finite terms or a finite sketch of finite lines. Finite terms and lines are signs. What they indicate need not be finite but what they are, must be.
Fragments —
The common sensorium.
The common sense and the senses of “common”.
This is the point where the empirical and the rational meet.
I describe as “empirical” any method which exposes theoretical descriptions of an object to further experience with that object.
Overview
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_OverviewObstacles
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_Part_5#Obstacles#Peirce #Inquiry #InquiryIntoInquiry #InquiryDrivenSystems
#Semiotics #SignRelations #Semiositis #ObstaclesToInquiry
#Logic #Abduction #Deduction #Induction #ScientificMethod
#Experience #Expectation #EffectiveDescription #FiniteMeans
#Abstraction #Analogy #Form #Matter #Empiricism #Rationalism -
In the Way of Inquiry • Material Exigency 1
• https://inquiryintoinquiry.com/2023/01/20/in-the-way-of-inquiry-material-exigency-a/Our survey of obstacles to inquiry has dealt at length with blocks arising from its formal aspects. On the other hand, I have cast this project as an empirical inquiry, proposing to represent experimental hypotheses in the form of computer programs. At the heart of that empirical attitude is a feeling all formal theories should arise from and bear on experience.
Every season of growth in empirical knowledge begins with a rush to the sources of experience. Every fresh‑thinking reed of intellect is raised to pipe up and chime in with the still‑viable canons of inquiry in one glorious paean to the personal encounter with natural experience.
But real progress in the community of inquiry depends on observers being able to orient themselves to objects of common experience — the uncontrolled exaltation of individual phenomenologies leads as a rule to the disappointment and disillusionment which befalls the lot of unshared enthusiasms and fragmented impressions.
Look again at the end of the season and see it faltering to a close, with every novice scribe rapped on the knuckles for departing from that uninspired identification with impersonal authority which expresses itself in third‑person passive accounts of one's own experience.
Overview
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_OverviewObstacles
• https://oeis.org/wiki/Inquiry_Driven_Systems_%E2%80%A2_Part_5#Obstacles#Peirce #Inquiry #InquiryIntoInquiry #InquiryDrivenSystems
#Semiotics #SignRelations #Semiositis #ObstaclesToInquiry
#Logic #Abduction #Deduction #Induction #ScientificMethod
#Abstraction #Analogy #Form #Matter #Empiricism #Rationalism
