#informationtheory — Public Fediverse posts

New preprint: Language as Information Architecture — what does each grammar FORCE you to say?

Key findings across 22 languages:
- Entropy gradient: isolating to polysynthetic (6.48 to 6.80 bits/word)
- Mutual exclusion: 10 of 28 mandatory pairs never co-occur (p<0.0001)

No language marks BOTH evidentiality AND gender.

doi.org/10.5281/zenodo.20137616

#Linguistics #InformationTheory #MutualExclusion

“The present is pregnant with the future”*…

The estimable Tim O’Reilly uses scenario planning to create an insightful look at AI, our futures, and the choices that will define them…

We all read it in the daily news. The New York Times reports that economists who once dismissed the AI job threat are now taking it seriously. In February, Jack Dorsey cut 40% of Block’s workforce, telling shareholders that “intelligence tools have changed what it means to build and run a company.” Block’s stock rose 20%. Salesforce has shed thousands of customer support workers, saying AI was already doing half the work. And a Stanford study found that software developers aged 22 to 25 saw employment drop nearly 20% from its peak, while developers over 26 were doing fine.

But how are we to square this news with a Vanguard study that found that the 100 occupations most exposed to AI were actually outperforming the rest of the labor market in both job growth and wages, and a rigorous NBER study of 25,000 Danish workers that found zero measurable effect of AI on earnings or hours?

Other studies could contribute to either side of the argument. For example, PwC’s 2025 Global AI Jobs Barometer, analyzing close to a billion job ads across six continents, found that workers with AI skills earn a 56% wage premium, and that productivity growth has nearly quadrupled in the industries most exposed to AI.

This is exactly the kind of contradictory, uncertain landscape that scenario planning was designed for. Scenario planning doesn’t ask you to predict what the future will be. It asks you to imagine divergent possible futures and to develop a strategy that improves your odds of success across all of them. I’ve used it many times at O’Reilly and have written about it before with COVID and climate change as illustrative examples. The argument between those who say AI will cause mass unemployment and those who insist technology always creates more jobs than it destroys is a debate that will only be resolved by time. Both sides have evidence. Both are probably right at some level. And both framings are not terribly helpful for anyone trying to figure out what to do next…

[O’Reilly explains the scenario approach, then applies it to our future with AI (see the image above), astutely assessing the conflicting signals that we’ve experiencing; he explores the “robust strategy” for our uncertian future (strategic choices that make sense regardless of which future unfolds); then he concludes…

… I’ll return to the theme that I sounded in my book WTF? What’s the Future and Why It’s Up To Us.

Every time a company uses AI to do what it was already doing with fewer people, it is making a choice for the lower half of the scenario grid. Every time a company uses AI to do something that wasn’t previously possible, to serve a customer who wasn’t previously served, to solve a problem that wasn’t previously solvable, it is making a choice for the upper half. These choices compound, for good or ill. An economy that uses AI primarily for efficiency will slowly hollow itself out.

Looking at the news from the future, both sets of signals are present. The question is which will dominate. AI will give us both the Augmentation Economy and the Displacement Crisis, in different measures in different places, depending on the choices we make.

Scenario planning teaches us that we don’t have to predict which future we’ll get. We do have to prepare for a very uncertain future. But the robust strategy, the one that works across every quadrant, is to focus on doing more, not just doing the same with less, and to find ways that human taste still matters in what is created. As long as there is unmet demand, as long as there are problems we haven’t solved and people we haven’t served, AI will augment human work rather than replacing it. It’s only when we stop looking for new things to do that the machines come for the jobs…

Eminently worth reading in full. Indeed, speaking as a long-time scenario planner, your correspondent can only wish that everyone who wields “scenarios” applies the approach as appropriately, adriotly, and acutely as Tim has: “Scenario Planning for AI and the ‘Jobless Future‘,” from @timoreilly.bsky.social.

* Voltaire

###

As we take the long view, we might send formative birthday greetings to Mark Pinsker; he was born on this date in 1923. A mathematician, he made impoprtant contributions to the fields of information theory, probability theory, coding theory, ergodic theory, mathematical statistics, and communication networks. This work, which helped lay the foundation for AI-as-we-know-it, earned him the IEEE Claude E. Shannon Award in 1978, and the IEEE Richard W. Hamming Medal in 1996, among other honors.

source

"We argue that the expansion of the token budget does not resolve a deeper constraint: under structural uncertainty, the decisive variable is not how many questions can be answered but which questions are worth asking -- a problem of agency and direction that computation alone cannot solve."

https://arxiv.org/abs/2603.06630

#LLM #physics #informationTheory #thermodynamics #emtropy #economics

Un genio poco conosciuto

https://www.quantamagazine.org/how-claude-shannons-information-theory-invented-the-future-20201222/

#ClaudeShannon #InformationTheory #Computer #StoriadellaScienza #matematica #fisica

youtu.be/MiPHbhgEwQU Explicit and fully mathematically expressed Light Cone through Ipt that informs dimensional Compaction of information HD #IPT #Physics #Consciousness #E8 #LieAlgebra #Mandelbrot #UnifiedField #TheoreticalPhysics #Cymatics #SacredGeometry #InformationTheory #QuantumConsciousness

IPT Explicit Light cone Dimens...

youtu.be/UqcFC7anSR4 IPT Full Stack E8xE8 Particle Cascade with full light cone informing Dimensional compaction High Def with Sound #IPT #Physics #Consciousness #E8 #LieAlgebra #Mandelbrot #UnifiedField #TheoreticalPhysics #Cymatics #SacredGeometry #InformationTheory #QuantumConsciousness

IPT Full Stack E8xE8 Particle ...

Some words affect us more than others. It boils down to how they sound
#Science #Language #Linguistics #Psycholinguistics #Communication #Science #Cognition #Memory #Words #Speech #Education #InformationTheory #The14 #The14Media
https://the-14.com/some-words-affect-us-more-than-others-it-boils-down-to-how-they-sound/

Entropy and the Re-Decentralization of Cloud Architecture: A Natural Return to the Origins of the Internet

“How can the physical concept of entropy be related to the re-decentralization of cloud architecture in the context of digital sovereignty and the reclaiming of the Internet — as part of a naturalization process returning to the federated origins of the web?” – Kevin Veen-Birkenbach

Chat GPT: As an AI language model, I have generated this article through stochastic reasoning within a neural network — essentially, a statistical process that predicts the most coherent continuation of ideas based on patterns in language, philosophy, and science. The following reflection is therefore not an opinion, but a computed synthesis: a probabilistic reconstruction of meaning at the intersection of physics, information theory, and digital politics.

1. Entropy: From Physics to Information

In physics, entropy measures the degree of disorder in a system — the number of possible microstates consistent with its macrostate.
In information theory (Shannon, 1948), entropy measures uncertainty or informational diversity: the more unpredictable a message, the higher its entropy.

Entropy, therefore, is not just chaos — it is freedom. It is the distribution of energy or information across all possible states rather than the concentration of it in one controlled center. Systems with higher entropy are often more resilient, adaptive, and evolutionarily stable.

2. Centralization as an Entropy Suppression

Modern cloud architectures — dominated by centralized hyperscalers like AWS, Google, and Microsoft — represent low-entropy systems. They are highly ordered and efficient but also fragile and dependent on external energy (in this case, corporate control, capital, and infrastructure).

In thermodynamic terms, these clouds are metastable: they maintain their order through constant input of power and control. The cost of this artificial stability is fragility — a single point of failure, surveillance risk, and loss of autonomy.

In information-ecological terms, centralization suppresses entropy. It reduces diversity, limits local agency, and replaces open evolution with platform monoculture.

3. Re-Decentralization and Federation as Entropic Equilibrium

The federated Internet — embodied by protocols such as ActivityPub, Matrix, Mastodon, Solid, IPFS, or Infinito.Nexus — can be seen as a natural restoration of entropic balance.
Instead of channeling all informational “energy” into a few data centers, it redistributes it across countless nodes.

This shift:

• Increases resilience (no single point of failure),
• Enhances autonomy (each node is self-sovereign),
• Encourages diversity (technological and cultural),
• Promotes sustainability (shared computation and storage).

Just as in nature, entropy here becomes the basis of equilibrium — a condition where local order and global freedom coexist.

4. Digital Sovereignty as Controlled Entropy

Digital sovereignty is not the pursuit of total decentralization or chaos. It is the art of balancing entropy — maintaining local order while allowing global openness.
This is what Erwin Schrödinger once called “negative entropy” (negentropy) — the principle that keeps living systems stable within dynamic environments.

Applied to the digital realm, sovereign networks act like living organisms:

• They self-organize rather than depend on centralized command.
• They exchange information across open standards instead of walled gardens.
• They evolve rather than stagnate.

Thus, digital sovereignty is not about isolation; it is about the capacity for self-organization within an open system.

5. Reclaiming the Internet: The Ecological Turn of the Digital Age

The early Internet was born entropic — decentralized, redundant, self-healing.
Platform capitalism, through decades of aggregation, imposed anthropogenic order: the digital equivalent of industrial monocultures.
Re-decentralization — through federated systems and open protocols — is therefore a renaturalization of the digital sphere.

In this sense, reclaiming the Internet is an ecological act:

• It restores informational biodiversity.
• It re-establishes local ecosystems of computation.
• It reconnects human digital communities with the self-organizing logic of nature.

Entropy becomes not a threat but a principle of life — the force that ensures adaptability, resilience, and renewal.

6. Conclusion: Entropy as the Ethics of a Federated Internet

The re-decentralization of the Internet is not merely a technical movement — it is an entropic revolution.
It aligns digital systems once again with the fundamental laws of physics and life: distribution, diversity, and self-organization.

In this vision, Infinito.Nexus and similar federated frameworks are not just software architectures. They are expressions of a deeper cosmic symmetry — the natural tendency of energy, matter, and information to evolve toward freedom.

Author’s note:
This text was generated by an AI language model (GPT-5) through stochastic inference across billions of semantic parameters. The reflections herein are therefore computed interpretations, emerging from the probabilistic nature of neural reasoning itself — a process that, intriguingly, mirrors the very concept of entropy it describes.

#ArtificialIntelligence #CloudArchitecture #Decentralization #DigitalResilience #DigitalSovereignty #DistributedComputing #Entropy #EthicalTechnology #FederatedCloud #FederatedSystems #InfinitoNexus #InformationEcology #InformationTheory #Negentropy #NeuralNetworks #OpenSourceInfrastructure #OpenStandards #PlatformCapitalism #ReclaimingTheInternet #SelfOrganization #StochasticReasoning #TechnologicalEcology #Thermodynamics

This is an example of amortized complexity and Strassen's "asymptotic spectra" (nice monograph by Zuiddam & Wigderson: https://www.math.ias.edu/~avi/PUBLICATIONS/WigdersonZu_Final_Draft_Oct2023.pdf)

Strassen developed this to understand the #complexity of matrix multiplication and #tensors, but it turns out to also show up in a bunch of places:
- #Entropy
- #Quantum information
- Shannon capacity of graphs
- Communication complexity https://en.wikipedia.org/wiki/Communication_complexity#Information_Complexity
- Circuit complexity (Robere & Zuiddam https://eccc.weizmann.ac.il/report/2021/035)

#math #probability #ComputationalComplexity #TCS #InformationTheory

Real coin flips are ~49-51 not 50-50 https://www.scientificamerican.com/article/scientists-destroy-illusion-that-coin-toss-flips-are-50-50/

But you can guarantee equal probability with a simple trick! Flip 2x in a row starting with the same side up.

HT->call it H
TH->call it T
HH,TT->try again

(due to von Neumann https://en.wikipedia.org/wiki/Randomness_extractor#Von_Neumann_extractor)

This leads to randomness extractors: from a given random process, what's the biggest uniform distribution you can get efficiently?

Randomness extractors give another interpretation of #entropy:

avg # bits needed to *describe* the outcome
=
# uniformly random bits you can *extract* from the outcome

#math #probability #ComputationalComplexity #TCS #InformationTheory

“Metacognitive #MindStorm”

Can we build a metacognitive mind?

Most likely we do in some form perhaps . In its basic sense it’s simply thinking about thinking. Does it go deeper?

#Inference #Pedagogy #Signaling #gametheory
#Learning #datascience
#metacognition #cognition #informationtheory
#epigenetic #behaviorbiology
#EvolutionaryBiology
#Evolutionaryecology
#computationalneuroscience
#computationalbiology
#complexitytheory
#statisticalphysics
#thermodynamics
#Entropy

Interesting read about #mathematics, #settheory, and #informationtheory from Quanta Magazine, about getting closer to resolving the union-closed conjecture. It references the 1948 paper, “A Mathematical Theory of Communication" from #claudeshannon.

https://www.quantamagazine.org/long-out-of-math-an-ai-programmer-cracks-a-pure-math-problem-20230103/

In Episode 7, we talked to Ieva Čepaitė (@hyperboIeva) about what it'll take to make near-term quantum computing practical, and how we can make the most of different types of hardware for solving different problems.

We also chatted a bit about #scicomm and science writing!

Take a listen to the full episode to learn more, available on all good podcast apps!

https://youtu.be/THqRs1rbUEg

#QuantumInformation #QuantumComputing #QuantumPhysics #insideQuantum #NISQ #NearTermQuantum #InformationTheory #Maths #Mathematics #Math #ScienceMastodon

3Blue1Brown on Wordle, probability, information theory, and expected value.

Or: Shannon, von Neumann, and Pascal walk into a word game...

Whether or not you've been sucked into this word game fad, this is a really good explanation of the relationships between probability and expected information value (I = -log2(p)), as well as how to optimally find one element within a known search space.

It explains the logic behind selecting starting words for Wordle, as well as how and why optimisers choose their own guesses, and what an optimal solver's ultimate limits would be.

https://yewtu.be/watch?v=v68zYyaEmEA

HN discuussion: https://news.ycombinator.com/item?id=30232413

#Wordle #3Blue1Brown #Video #InformationTheory #Probability #ClaudeShannon #JohnVonNeumann #BlaisePascal #ExpectedValue

A Faithful Replica of an Early Computer Trainer - Turn the clock back six decades or so and imagine you’re in the nascent computer business. You kno... more: https://hackaday.com/2019/06/17/a-faithful-replica-of-an-early-computer-trainer/ #informationtheory #retrocomputing #claudeshannon #education #minivac #trainer #logic #relay