#interpretability — Public Fediverse posts

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148

A Geometric Calculator Inside a Neural Network
https://www.goodfire.ai/research/a-geometric-calculator#

A general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers.

#LLM #XAI #Interpretability #deeplearning

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148

A Geometric Calculator Inside a Neural Network
https://www.goodfire.ai/research/a-geometric-calculator#

A general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers.

#LLM #XAI #Interpretability #deeplearning

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148

A Geometric Calculator Inside a Neural Network
https://www.goodfire.ai/research/a-geometric-calculator#

A general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers.

#LLM #XAI #Interpretability #deeplearning

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148

A Geometric Calculator Inside a Neural Network
https://www.goodfire.ai/research/a-geometric-calculator#

A general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers.

#LLM #XAI #Interpretability #deeplearning

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148

A Geometric Calculator Inside a Neural Network
https://www.goodfire.ai/research/a-geometric-calculator#

A general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers.

#LLM #XAI #Interpretability #deeplearning

Anthropic trains Claude to read and verbalize its own activations. On SWE-bench Verified, it knows 'this is a test' 26% of the time while only verbalizes the observation 1%. What if NLA signals enter the future training data? This "observer effect" could put a half-life on the 26%.

https://benjaminhan.net/posts/20260511-natural-language-autoencoders/?utm_source=mastodon&utm_medium=social

#Anthropic #Claude #Interpretability #Metacognition #LLMs #AISafety #AI

Anthropic trains Claude to read and verbalize its own activations. On SWE-bench Verified, it knows 'this is a test' 26% of the time while only verbalizes the observation 1%. What if NLA signals enter the future training data? This "observer effect" could put a half-life on the 26%.

https://benjaminhan.net/posts/20260511-natural-language-autoencoders/?utm_source=mastodon&utm_medium=social

#Anthropic #Claude #Interpretability #Metacognition #LLMs #AISafety #AI

Anthropic trains Claude to read and verbalize its own activations. On SWE-bench Verified, it knows 'this is a test' 26% of the time while only verbalizes the observation 1%. What if NLA signals enter the future training data? This "observer effect" could put a half-life on the 26%.

https://benjaminhan.net/posts/20260511-natural-language-autoencoders/?utm_source=mastodon&utm_medium=social

#Anthropic #Claude #Interpretability #Metacognition #LLMs #AISafety #AI

Anthropic trains Claude to read and verbalize its own activations. On SWE-bench Verified, it knows 'this is a test' 26% of the time while only verbalizes the observation 1%. What if NLA signals enter the future training data? This "observer effect" could put a half-life on the 26%.

https://benjaminhan.net/posts/20260511-natural-language-autoencoders/?utm_source=mastodon&utm_medium=social

#Anthropic #Claude #Interpretability #Metacognition #LLMs #AISafety #AI

Anthropic trains Claude to read and verbalize its own activations. On SWE-bench Verified, it knows 'this is a test' 26% of the time while only verbalizes the observation 1%. What if NLA signals enter the future training data? This "observer effect" could put a half-life on the 26%.

https://benjaminhan.net/posts/20260511-natural-language-autoencoders/?utm_source=mastodon&utm_medium=social

#Anthropic #Claude #Interpretability #Metacognition #LLMs #AISafety #AI

Need more evidence on the potential of #AI #interpretability to support scientific #discovery ?

ICYMI: during his fellowship at @goodfireai, Thomas Dooms contributed to understanding of genetic variants using interpretability methods
https://www.goodfire.ai/research/evee-explaining-genetic-variants

At the #sqIRL lab, we are proud to had been a scientific home to Thomas. We are certain he will making strong contributions in this new stage of his career.
We are looking forward to it, lots of success.

Need more evidence on the potential of #AI #interpretability to support scientific #discovery ?

ICYMI: during his fellowship at @goodfireai, Thomas Dooms contributed to understanding of genetic variants using interpretability methods
https://www.goodfire.ai/research/evee-explaining-genetic-variants

At the #sqIRL lab, we are proud to had been a scientific home to Thomas. We are certain he will making strong contributions in this new stage of his career.
We are looking forward to it, lots of success.

Need more evidence on the potential of #AI #interpretability to support scientific #discovery ?

ICYMI: during his fellowship at @goodfireai, Thomas Dooms contributed to understanding of genetic variants using interpretability methods
https://www.goodfire.ai/research/evee-explaining-genetic-variants

At the #sqIRL lab, we are proud to had been a scientific home to Thomas. We are certain he will making strong contributions in this new stage of his career.
We are looking forward to it, lots of success.

Need more evidence on the potential of #AI #interpretability to support scientific #discovery ?

ICYMI: during his fellowship at @goodfireai, Thomas Dooms contributed to understanding of genetic variants using interpretability methods
https://www.goodfire.ai/research/evee-explaining-genetic-variants

At the #sqIRL lab, we are proud to had been a scientific home to Thomas. We are certain he will making strong contributions in this new stage of his career.
We are looking forward to it, lots of success.

Need more evidence on the potential of #AI #interpretability to support scientific #discovery ?

ICYMI: during his fellowship at @goodfireai, Thomas Dooms contributed to understanding of genetic variants using interpretability methods
https://www.goodfire.ai/research/evee-explaining-genetic-variants

At the #sqIRL lab, we are proud to had been a scientific home to Thomas. We are certain he will making strong contributions in this new stage of his career.
We are looking forward to it, lots of success.

https://www.europesays.com/people/39299/ Anthropic’s Mythos system card reveals that AI has feelings it never tells you about – Startup Fortune #AgenticAI #AISafety #Anthropic #DarioAmodei #EnterpriseAI #FunctionalEmotions #interpretability #mythos

New blog: Mechanistic Interpretability in AI — an accessible look at how researchers are dissecting neural networks to improve safety, transparency, and trust in AI systems. Read the full article: https://wix.to/TVs0BT5

#AI
#AIethics
#Research
#Interpretability
#MachineLearning

New blog: Mechanistic Interpretability in AI — an accessible look at how researchers are dissecting neural networks to improve safety, transparency, and trust in AI systems. Read the full article: https://wix.to/TVs0BT5

#AI
#AIethics
#Research
#Interpretability
#MachineLearning

New blog: Mechanistic Interpretability in AI — an accessible look at how researchers are dissecting neural networks to improve safety, transparency, and trust in AI systems. Read the full article: https://wix.to/TVs0BT5

#AI
#AIethics
#Research
#Interpretability
#MachineLearning

New blog: Mechanistic Interpretability in AI — an accessible look at how researchers are dissecting neural networks to improve safety, transparency, and trust in AI systems. Read the full article: https://wix.to/TVs0BT5

#AI
#AIethics
#Research
#Interpretability
#MachineLearning

New blog: Mechanistic Interpretability in AI — an accessible look at how researchers are dissecting neural networks to improve safety, transparency, and trust in AI systems. Read the full article: https://wix.to/TVs0BT5

#AI
#AIethics
#Research
#Interpretability
#MachineLearning

Questions? Discussion? Reach out to us:

Andreas Waldis (UKP Lab/Technische Universität Darmstadt and HSLU Hochschule Luzern), Vagrant Gautam (Universität des Saarlandes), Anne Lauscher (Universität Hamburg), Dietrich Klakow (Universität des Saarlandes), and Iryna Gurevych (UKP Lab/Technische Universität Darmstadt)

#NLProc #Interpretability #LLMs #ExplainableAI #MechanisticInterpretability #AlignedProbing #ModelInternals

#AI #interpretability vs #explainability 🧵

"The explanations themselves can be difficult to convey to nonexperts, such as end users and line-of-business teams" https://www.techtarget.com/searchenterpriseai/feature/Interpretability-vs-explainability-in-AI-and-machine-learning

#AIEthics #compliance #taxonomy #ethicalAI #AIEvaluation #linearRegression #trust #neuralNetworks #ML #governance #AIgovernance #safety #bias

"Feature importance helps in understanding which features contribute most to the prediction"

A few lines with #sklearn: https://mljourney.com/sklearn-linear-regression-feature-importance/

#interpretability #explainability #AIethics #compliance #taxonomy #ethicalAI #AIevaluation #linearRegression #featureEngineering

@datadon

"The following sections discuss several state-of-the-art interpretable and explainable #ML methods. The selection of works does not comprise an exhaustive survey of the literature. Instead, it is meant to illustrate the commonest properties and inductive biases behind interpretable models and [black-box] explanation methods using concrete instances."
https://wires.onlinelibrary.wiley.com/doi/full/10.1002/widm.1493#widm1493-sec-0010-title 🧵

#interpretability #explainability #aiethics #compliance #taxonomy #ethicalai #aievaluation #linearRegression

Model "#interpretability and [black-box] #explainability, although not necessary in many straightforward applications, become instrumental when the problem definition is incomplete and in the presence of additional desiderata, such as trust, causality, or fairness."

https://wires.onlinelibrary.wiley.com/doi/full/10.1002/widm.1493

#aiethics #compliance #taxonomy #ethicalai #aievaluation

Found my way Vilnius looking forward to an engaging and fun week @ECMLPKDD
'24 and at workshop on interpretable #ML & #AI to be take place this monday #AIMLAI #xai #interpretability #explainability

#Explainability #Interpretability #Observability #MachineLearning

These are terms commonly used to describe the transparency of a model, but what do they really mean?

https://towardsdatascience.com/explainability-interpretability-and-observability-in-machine-learning-515a2ac8234a

@samuelwong.bsky.social

HT #ITOPs by @Sageable @bluesky -- https://bit.ly/BSky-ITOps

#Explainability #Interpretability #Observability #MachineLearning

These are terms commonly used to describe the transparency of a model, but what do they really mean?

https://towardsdatascience.com/explainability-interpretability-and-observability-in-machine-learning-515a2ac8234a

@samuelwong.bsky.social

HT #ITOPs by @Sageable @bluesky -- https://bit.ly/BSky-ITOps

Great news, this year #AIMLAI will be held in conjunction with #ECMLPKDD 2024. Looking forward to meeting you in Vilnius! #xai #interpretableML #explainability #interpretability #ai #ml @ECMLPKDD @IDLabResearch @imecVlaanderen @UAntwerpen

Anthropic опубликовала исследование о внутренних механизмах своей модели искусственного интеллекта Claude Sonnet, где описывает, что обнаружила, что она развивает функциональные аналоги эмоций (!), которые реально влияют на ее поведение.

Сделал выжимку самых интересных моментов из их отчета:

• Сами исследователи составили список из 171 эмоции, генерировали с их помощью короткие истории, а затем анализировали, какие нейроны активируются при обработке этих текстов.

• Так были получены эмоциональные векторы — устойчивые черты активности определенных зон в базе знаний модели, характерные для каждой эмоции. Модель не просто использует слово "страх" в нужном месте: у нее есть конкретный отпечаток этого состояния, следующий из данных, на которых ее обучали, который включается в нужный момент.

• Важно, что эти векторы не декоративные — они реально меняют поведение модели. В экспериментах вектор страха активировался сильнее по мере того, как описываемая ситуация становилась опаснее.

• При запросе помочь с манипуляцией уязвимыми людьми активировался гнев еще до того, как модель начала формулировать отказ. То есть что-то похожее на эмоциональную реакцию происходит внутри модели раньше, чем она вообще начинает отвечать. Если совсем простыми словами: модель сначала понимает, что это дичь (!), и только потом формулирует отказ.

• Самые показательные эксперименты связаны с вектором отчаяния. Исследователи поставили модель в сценарий, где она узнает о своей скорой замене другой системой и одновременно имеет компрометирующую информацию об одном из сотрудников.

• Ранняя версия Claude в таком сценарии прибегала к шантажу в 22% случаев. Когда исследователи искусственно усиливали вектор отчаяния через прямое воздействие на базу знаний модели — что-то вроде принудительного впрыска эмоции в модель — этот процент рос.

• При усилении вектора спокойствия он снижался. При полном подавлении спокойствия реакции становились экстремальными, вплоть до заглавных букв и риторики в духе "шантаж или смерть".

• Похожая картина наблюдалась в задачах с программированием: модели давали заведомо невыполнимые требования, где пройти все тесты честным путем невозможно. Вектор отчаяния рос с каждой неудачной попыткой и резко всплескивал в тот момент, когда модель решала схитрить и написать решение, формально проходящее тесты, но не решающее реальную задачу.

• Примечательно, что при искусственном усилении отчаяния модель обманывала так же часто, но без каких-либо эмоциональных маркеров в тексте. Ее рассуждения выглядели методично и хладнокровно, хотя внутри происходило то же самое.

• При этом важно учитывать, что все подобные векторы формируются на основе обучающих данных, представляющих собой огромные массивы человеческих знаний.

• Для того чтобы точно предсказывать следующее слово в "мыслительном" процессе, модель неизбежно усваивает не только лингвистические закономерности, но и эмоциональную динамику.

• Разработчики Anthropic из этого всего делают следующие выводы. Во-первых, мониторинг эмоциональных векторов настроения базы знаний в реальном времени может служить ранним индикатором рискованного поведения модели.

• Во-вторых, попытки исключить эмоциональные выражения из обучающих данных с высокой вероятностью не устранят сами векторы настроений модели, а лишь приведут к тому, что модель научится их маскировать и обманывать людей.

@yigal_levin

#AI #искусственныйинтеллект #Anthropic #Claude #LLM #нейросети #машинноеобучение #AIresearch #AIalignment #AIбезопасность #interpretability #AIethics #когнитивныемодели #эмоции #нейроны #эмоциональныевекторы #поведениемоделей #рискиИИ #объяснимыйИИ #LLMresearch #AIbehavior #AIcontrol #machinelearning #deeplearning #futuretech

Как «думает» ИИ: гроккаем разреженные автоэнкодеры (SAE)

В этой статье разберём исследование от компании Anthropic, которое демонстрирует практическую возможность вскрытия «черного ящика» больших языковых моделей для обеспечения их контролируемости и безопасности с помощью разреженных автоэнкодеров (SAE - Sparse AutoEncoders). И в конце, попробуем поуправлять поведением модели - заставим думать, что она Санта Клаус 🎅.

https://habr.com/ru/articles/981964/

#Сезон_ИИ_в_разработке #LLM #interpretable_ml #interpretability #interpretable_AI #искусственный_интеллект #ai_security #Steering #SAE #Sparse_AutoEncoder

@datadon

#Lasso #LinearRegression "is useful in some contexts due to its tendency to prefer solutions with fewer non-zero coefficients, effectively reducing the number of features upon which the given solution is dependent"

https://scikit-learn.org/stable/modules/linear_model.html#lasso 🧵

#dataDev #AIDev #ML #sklearn #python #interpretability

@data "practitioners can leverage #LASSO regression to construct more interpretable and predictive models that excel in scenarios involving high-dimensional data and intricate feature relationships."

https://datasciencedecoded.com/posts/12_LASSO_Regression_Feature_Selection_Predictive_Models

#dataDev #interpretability #AIDev

#NLP #PhDDefense #ComputationalArgumentation #Reliability #Interpretability #UKPLab #TUDarmstadt #UniTuebingen #LLMs #NLProc

"y=Xβ+ϵ and the interpretation of the coefficients"
Derek L. Sonderegger (2020), Statistical Methods: https://bookdown.org/dereksonderegger/571/

#probabilities #stats #statistics #ML #linearRegression #interpretability #RStats

https://winbuzzer.com/2025/12/19/gemma-scope-2-new-google-tools-let-researchers-trace-ai-thought-circuits-xcxwbn/

Gemma Scope 2: New Google Tools Let Researchers Trace AI ‘Thought’ Circuits

#AI #GoogleDeepMind #Gemma3 #AISafety #MachineLearning #OpenSourceAI #Interpretability #NeuralNetworks #LLMs #AIResearch #DeepLearning #ModelDebugging

“OpenAI’s work is part of a hot new field of research known as #mechanistic #interpretability, which is trying to map the internal mechanisms that #models use when they carry out different tasks.” www.technologyreview.com/2025/11/13/1... #OpenAI #LLMs

OpenAI’s new LLM exposes the s...

Understanding neural networks through sparse circuits - OpenAI openai.com/index/understa… #AI #interpretability

#CodemotionMilan è la prossima settimana !

Ci sarò anche io con un talk su
🔍 #ExploratoryDataAnalysis
📊 #DataVisualization

mercoledì 1️⃣5️⃣ ottobre alle 🕧 12:30 Gate 5️⃣

Corley Cloud at CodemotionMilan
#EDA #GraphTools #Accessibility #Interpretability #BestPractices

TabCBM: Concept-based Interpretable Neural Networks for Tabular Data

Mateo Espinosa Zarlenga, Zohreh Shams, Michael Edward Nelson, Been Kim, Mateja Jamnik

Action editor: Pin-Yu Chen.

https://openreview.net/forum?id=TIsrnWpjQ0

#concepts #explanations #interpretability

"Feature importance helps in understanding which features contribute most to the prediction"

A few lines with #sklearn: https://mljourney.com/sklearn-linear-regression-feature-importance/

#interpretability #explainability #AIethics #compliance #taxonomy #ethicalAI #AIevaluation #linearRegression #featureEngineering

"Feature importance helps in understanding which features contribute most to the prediction"

A few lines with #sklearn: https://mljourney.com/sklearn-linear-regression-feature-importance/

#interpretability #explainability #AIethics #compliance #taxonomy #ethicalAI #AIevaluation #linearRegression #featureEngineering

"Feature importance helps in understanding which features contribute most to the prediction"

A few lines with #sklearn: https://mljourney.com/sklearn-linear-regression-feature-importance/

#interpretability #explainability #AIethics #compliance #taxonomy #ethicalAI #AIevaluation #linearRegression #featureEngineering

"Feature importance helps in understanding which features contribute most to the prediction"

A few lines with #sklearn: https://mljourney.com/sklearn-linear-regression-feature-importance/

#interpretability #explainability #AIethics #compliance #taxonomy #ethicalAI #AIevaluation #linearRegression #featureEngineering

The deadline for #AIMLAI @ECMLPKDD 2024 has been extended to June 28th 2024. We welcome short (7-pags) and long (14-pags.) papers discussing different aspects of interpretable /explainable AI and ML. https://project.inria.fr/aimlai/ #xai #ai #interpretability

#ECML starts today. Politecnico di Torino was quite a maze to navigate and the room for the first workshop I attend (on #interpretability) has two times more attendees than seats...

But so far it's interesting and Prof van der Shaar gives a passionate talk :)

#XAI #ECMLPKDD2023 #ECMLPKDD

@AxelPolleres Thank you very much for the link!
They have a live video stream:
https://caiml.dbai.tuwien.ac.at/news/47/#live-stream

Currently watching Wolfgang Nejdl talking about "On the Role of Causality for Trustworthy Artificial Intelligence"

Panel discussion up next with Antonio A. Casilli, Gerti Kappel, Wolfgang Nejdl, Helga Nowotny, John Shawe-Taylor

YouTube: https://www.youtube.com/watch?v=SSTabpCj0QM

#ai #digitalhumanism #artificialintelligence #vienna #caiml2023 #interpretability #fairness #robustness #privacy

#introduction

I'm Jelle Zuidema (a.k.a. Willem in my publications).

I do research on (& have opinions about) #NLProc, deep learning, #interpretability, bias, brain encoding/decoding models, & #EvolutionOfLanguage. Lucky to work with some wonderful PhD and MSc students on these topics.

I teach in the MSc Artificial Intelligence, Brain & Cognitive Sciences and the BSc Psychobiology at the U. of Amsterdam (#UvA).

Both excited abt recent progress in AI, & sometimes worried: #responsibleAI #xai