#differential-privacy — Public Fediverse posts

Our AAAI 2026 poster on private in-context learning: LLMs can use sensitive records as in-context examples, with formal differential privacy guarantees and no fine-tuning.

Poster: https://doi.org/10.48448/8as1-z797
Paper: https://arxiv.org/abs/2509.13625
Code: https://github.com/bhusalb/privacy-preserving-icl

#DifferentialPrivacy #LLM #AAAI2026 #Privacy

1/ Is your Gaussian mechanism actually (ε,δ)-DP? DP bugs are common and hard to spot by hand.

Our CCS 2025 paper shows that checking this for loop-free programs with Gaussian and Laplace sampling is almost decidable: decidable for every δ outside a finite set.

#DifferentialPrivacy #FormalVerification #Security

1/ Can an LLM use sensitive records as in-context examples without leaking them? Our AAAI 2026 paper: yes, with formal differential privacy guarantees and no fine-tuning.

Paper: https://arxiv.org/abs/2509.13625
Code: https://github.com/bhusalb/privacy-preserving-icl

#DifferentialPrivacy #LLM #Privacy

1/ Muon is quickly becoming a go-to optimizer for training LLMs. What happens when you train with it under differential privacy?

New preprint with collaborators at Los Alamos: Inner Momentum for Differentially Private Muon

https://arxiv.org/abs/2610.02738

#DifferentialPrivacy #MachineLearning #LLM

The Neighbor the Census Invented

https://bolesblogs.com/2026/09/28/the-neighbor-the-census-invented/

"The consequences will be dire for utility or for privacy, and possibly both. It's hard to understate this point: future statistical releases will either be useless compared to past ones, or they will be incredibly unsafe.

For starters, taking away useful tools from the disclosure avoidance toolbox will always lead to more painful privacy/utility trade-offs. The whole point of this research field is to better understand and quantify privacy risk, and develop better tools to mitigate this risk while preserving utility.

For statistical releases, differential privacy is simply the best tool we have right now. It provides a finer way of quantifying trade-offs, and allows us to get more utility out of the data than competing techniques at similar privacy levels. If you take it away, you're left with techniques that either have worse utility at similar levels of privacy, or worse privacy for the same utility.

But all competing techniques also rely on noise addition. The Cell Key method, used at other statistical agencies, adds noise to statistics. Swapping, used from 1990 to 2010 for the U.S. Census, also injects randomness into the process. Sampling is everywhere in statistical work2. Hell, even imputation technically adds noise to the data3!

By contrast, coarsening and suppression are very blunt instruments. They only work in situations where the statistics are already very coarse, and not too many of them are published.
(...)
It makes sense: privacy attacks on statistical releases are about solving a system of equations. It is such an easier task when you know for sure that the statistics are all perfectly accurate. Noise forces you to compute probabilities, quantify the uncertainty, carefully consider baselines, and so on. That's why randomness is such a useful tool for disclosure avoidance! Even without formal guarantees, it makes attakcs a lot harder. Take it away and attacks become trivial."
https://desfontain.es/blog/banning-noise.html
#USA #Census #Statistics #DifferentialPrivacy

US bans differential privacy in Census data

https://desfontain.es/blog/banning-noise.html

#HackerNews #USCensus #DifferentialPrivacy #DataPrivacy #PrivacyPolicy #Census2020

via @commercegov Disclosure Avoidance for Statistical Products | Order Number: DAO 216-26...
"Any use of noise infusion is inconsistent with the Department’s policies." https://www.commerce.gov/opog/disclosure-avoidance-statistical-products?utm_source=censusSDC #differentialprivacy

Hype for the Future 103F: Differential Privacy Defined

https://novatopflex.wordpress.com/2026/02/11/hype-for-the-future-103f-differential-privacy-defined/

This looks encouraging for privacy-preserving LLMs. While the actual differential privacy guarantees are notoriously difficult to interpret, "no memorisation" is a nice headline. Caveat: there is around 30% performance (utility) gap between the private and non-private models.

https://arxiv.org/abs/2510.15001

#privacy #ai #llm #differentialPrivacy

Giờ đây, bạn có thể chạy suy luận LLM cục bộ với bảo đảm quyền riêng tư chính thức! Một gói pip mới đã được phát hành, cho phép bạn sử dụng các mô hình ngôn ngữ lớn (LLM) trên thiết bị của mình với tính năng bảo mật dữ liệu mạnh mẽ thông qua suy luận riêng tư vi phân. Nâng cao quyền riêng tư cho người dùng LLM.
#LLM #Privacy #AI #LocalLLM #DifferentialPrivacy #QuyenRiengTu #MoHinhNgonNgu #BaoMatDuLieu

https://www.reddit.com/r/LocalLLaMA/comments/1puhjqk/now_you_can_run_local_llm_inference_with_

"Everyone sharing his or her data to train A.I. is great if we agree with the goals that were given to the A.I. It’s not so great if we don’t agree with these goals; and if the algorithm’s decisions might cost us our jobs, happiness, liberty or even lives.

To safeguard ourselves from collective harm, we need to build institutions and pass laws that give people affected by A.I. algorithms a voice over how those algorithms are designed, and what they aim to achieve. The first step is transparency. Similar to corporate financial reporting requirements, companies and agencies that use A.I. should be required to disclose their objectives and what their algorithms are trying to maximize — whether that’s ad clicks on social media, hiring workers who won’t join unions or total deportation counts.

The second step is participation. The people whose data are used to train the algorithms — and whose lives are shaped by them — should help decide their goals. Like a jury of peers who hear a civil or criminal case and render a verdict together, we might create citizens’ assemblies where a representative randomly chosen set of people deliberates and decides on appropriate goals for algorithms. That could mean workers at a firm deliberating about the use of A.I. at their workplace, or a civic assembly that reviews the objectives of predictive policing tools before government agencies deploy them. These are the kinds of democratic checks that could align A.I. with the public good, not just private power.

The future of A.I. will not be decided by smarter algorithms or faster chips. It will depend on who controls the data — and whose values and interests guide the machines. If we want A.I. that serves the public, the public must decide what it serves."

https://www.nytimes.com/2025/11/02/opinion/ai-privacy.html?unlocked_article_code=1.yU8.8BEa.DltbW_WwVhxN&smid=nytcore-android-share

#AI #Algorithms #Privacy #DifferentialPrivacy #AITraining

📬 US-Regierung will Anonymität der Volkszählung aufheben
#Datenschutz #Netzpolitik #CensusBureau #COUNTAct #Deanonymisierung #differentialprivacy #HowardLutnick #Staatsbürgerschaft #TopDown https://sc.tarnkappe.info/61d622

Republicans want to ban differential privacy in the Census, citing data distortion. But removing it could expose personal info, risking privacy for millions. It’s a battle over how we protect data in the age of AI.

https://www.wired.com/story/republicans-differential-privacy-census-overhaul/
#DataPrivacy #Census #Policy #Ethics #DifferentialPrivacy

The #Republican Plan to Reform the #Census Could Put Everyone’s #Privacy at Risk

A little-known #algorithmic process called “differential privacy” helps keep census data #anonymous. Conservatives want it gone.
#differentialprivacy #algorithms

https://www.wired.com/story/republicans-differential-privacy-census-overhaul/

🔏𝗥𝗲𝘃𝗶𝗲𝘄 𝗗𝗮𝘆 𝗧𝘂𝗲𝘀𝗱𝗮𝘆!📖

Differential Privacy is an increasingly popular, though controversial, technique for protecting confidential data by carefully introducing statistical noise. Even if you are very familiar with #DP, you likely don't know quite as much as Dr. Simson Garfinkel ( @xchatty )

This week, Ben Rothke ( @benrothke ) provides a Hall of Fame recommendation in his review of Garfinkel's new book, 𝘋𝘪𝘧𝘧𝘦𝘳𝘦𝘯𝘵𝘪𝘢𝘭 𝘗𝘳𝘪𝘷𝘢𝘤𝘺, from The MIT Press Essential Knowledge Series. ( @themitpress )

Review👉 https://tinyurl.com/2s3jk4cn

#CybersecurityBooks #CyberCanonHoFCandidate #DifferentialPrivacy #DataPrivacy

"This work on differential privacy has led to a new open-weight Google model called VaultGemma. The model uses differential privacy to reduce the possibility of memorization, which could change how Google builds privacy into its future AI agents. For now, though, the company's first differential privacy model is an experiment.

VaultGemma is based on the Gemma 2 foundational model, which is a generation behind Google's latest open model family. The team used the scaling laws derived from its initial testing to train VaultGemma with the optimal differential privacy. This model isn't particularly large in the grand scheme, clocking in at just 1 billion parameters. However, Google Research says VaultGemma performs similarly to non-private models of a similar size."

https://arstechnica.com/ai/2025/09/google-releases-vaultgemma-its-first-privacy-preserving-llm/

#AI #GenerativeAI #Google #VaultGemma #Chatbots #LLMs #Privacy #DifferentialPrivacy

An interesting #AI model from Google that could be very useful for #Biomedical & #Health applications, where you need to deal no only with #PII but also #PHI:

"VaultGemma: The world's most capable differentially private #LLM"

https://research.google/blog/vaultgemma-the-worlds-most-capable-differentially-private-llm/

#DifferentialPrivacy

Apple ujawnia trzy kluczowe badania z konferencji o prywatności i sztucznej inteligencji

Apple opublikowało prezentacje z Workshop on Privacy-Preserving Machine Learning (20–21 marca 2025), poświęconego prywatności i bezpieczeństwu w rozwoju AI.

Kilka miesięcy temu Apple zorganizowało warsztaty na temat uczenia maszynowego z zachowaniem prywatności, podczas których przedstawiono prezentacje i dyskusje na temat prywatności, bezpieczeństwa i innych kluczowych obszarów odpowiedzialnego rozwoju uczenia maszynowego. Teraz prezentacje te zostały upublicznione.

Podobnie jak niedawno w przypadku prezentacji z 2024 Workshop on Human-Centered Machine Learning, Apple opublikowało post na swoim blogu Machine Learning Research z kilkoma filmami i długą listą badań i artykułów, które zostały zaprezentowane podczas dwudniowego wydarzenia hybrydowego, które odbyło się w dniach 20-21 marca 2025 roku.

Do trzech najważniejszych prac należy zaliczyć.

Local Pan-Privacy for Federated Analytics – badanie Apple pokazuje, jak chronić prywatność danych nawet wtedy, gdy urządzenie zostanie wielokrotnie skompromitowane. Zastosowano nowe szyfrowane metody pozwalające zbierać statystyki bez ujawniania aktywności użytkownika.

Źródło: 9to5Mac.

Scalable Private Search with Wally – Apple zaprezentowało system wyszukiwania z wykorzystaniem differential privacy. Mechanizm Wally dodaje do zapytania losowe dane, co zapewnia anonimowość użytkowników, a jednocześnie umożliwia obsługę milionów żądań przy niższych kosztach.

Źródło: 9to5Mac.

Differentially Private Synthetic Data via Foundation Model APIs – badanie Microsoftu pokazuje, jak generować syntetyczne dane na podstawie modeli foundation, zachowując wartość danych rzeczywistych, ale bez naruszania prywatności.

Źródło: 9to5Mac.

Łącznie udostępniono 25 publikacji, przygotowanych przez badaczy z Apple, Microsoftu, Google oraz czołowych uczelni (m.in. MIT, UC Berkeley, Carnegie Mellon).

Oto ich pełna lista:

• AirGapAgent: Protecting Privacy-Conscious Conversational Agents
• A Generalized Binary Tree Mechanism for Differentially Private Approximation of All-Pair Distances
• Differentially Private Synthetic Data via Foundation Model APIs 1: Images
• Differentially Private Synthetic Data via Foundation Model APIs 2: Text
• Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy
• Elephants Do Not Forget: Differential Privacy with State Continuity for Privacy Budget
• Improved Differentially Private Continual Observation Using Group Algebra
• Instance-Optimal Private Density Estimation in the Wasserstein Distance
• Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
• Local Pan-privacy for Federated Analytics
• Nearly Tight Black-Box Auditing of Differentially Private Machine Learning
• On the Price of Differential Privacy for Hierarchical Clustering
• Operationalizing Contextual Integrity in Privacy-Conscious Assistants
• PREAMBLE: Private and Efficient Aggregation via Block Sparse Vectors
• Privacy amplification by random allocation
• Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss
• Privately Estimating a Single Parameter
• Scalable Private Search with Wally
• Shifted Composition I: Harnack and Reverse Transport Inequalities
• Shifted Interpolation for Differential Privacy
• Tractable Agreement Protocols
• Tukey Depth Mechanisms for Practical Private Mean Estimation
• User Inference Attacks on Large Language Models
• Universally Instance-Optimal Mechanisms for Private Statistical Estimation
• “What do you want from theory alone?” Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation

#AppleAI #AppleBadaniaAI #AppleKonferencjaPrywatność #bezpieczeństwoWAI #daneSyntetyczneAI #differentialPrivacy #federatedAnalytics #prywatnośćDanychApple #sztucznaInteligencjaApple #WallyApple

Differential privacy: Being wrong on purpose: How do you protect the #privacy of the subjects of #statistics and #data? – By adding controlled #noise. The blog Ironic Sans has an interesting and somewhat funny account of the ramifications of the application of #differentialprivacy in the...
https://spatialists.ch/posts/2025/07/01-differential-privacy-being-wrong-on-purpose/ #GIS #GISchat #geospatial #SwissGIS

Apple überarbeitet Siri: „LLM Siri“ soll Neuanfang bringen
Apple steht vor einem grundlegenden Umbau seines Sprachassistenten Siri. Ziel ist eine neue, KI-basierte Version namens „LLM Siri“. Diese Entwicklung folgt auf Problem
https://www.apfeltalk.de/magazin/news/apple-ueberarbeitet-siri-llm-siri-soll-neuanfang-bringen/
#KI #News #AppleAIStrategie #AppleIntelligence #AppleSprachassistent #DifferentialPrivacy #GenerativeKI #JohnGiannandrea #KIApple #LLMSiri #PerplexityAI #SiriUpdate

#AdaptiveCruiseControl(ACC) #AIgovernance #AIIntegration #augmentedreality #AutomatedDriving #AutonomousDriving #autonomousvehicles #autopilottechnology #BlockchainforSecurity #carsafety #Cloudcomputing #cybersecurity #DataAnonymization #Dataprivacy #deeplearning #DifferentialPrivacy #edgecomputing #encryption #ethicalAI #FailoverProtocols

https://miltonmarketing.com/computer-programming-languages/custom-code/revolutionizing-autonomy-introducing-a-next-generation-algorithm-for-autopilot-cars/

@benrothke.bsky.social has written a lovely review of #DifferentialPrivacy for his "book of the month" column at the RSAC Conference website. He notes that the book is available as an open source download from @mitpress.bsky.social .

https://www.rsaconference.com/library/blog/bens-book-of-the-month-differential-privacy

Fehlerbehaftete #AppleIntelligence: Abgleich mit #iPhone-Daten soll helfen | Mac & i https://www.heise.de/news/Fehlerbehaftete-Apple-Intelligence-Abgleich-mit-iPhone-Daten-soll-helfen-10353093.html #ArtificialIntelligence #Apple :apple_inc: #DifferentialPrivacy #Datenschutz #privacy

Apple details how it plans to analyze user data privately using differential privacy and synthetic data to improve its AI models. The approach will help enhance products like Genmoji, Image Playground, and more. #AppleAI #DifferentialPrivacy

#DifferentialPrivacy made the top position of @techreview's "Recent books from the MIT community" in the March/April edition.

https://www.technologyreview.com/2025/02/25/1111216/recent-books-from-the-mit-community-21/

Congratulations DOCTOR Liudas Panavas on the successful defense of his dissertation "Bridging the Gap: Human Centered Research for Democratizing #DifferentialPrivacy " 🎉 and congrats to advisor @codydunne ❤️ #HCI #DataVisualization

https://lpanavas.github.io/

I'm talking about #differentialPrivacy on the SecureTalk podcast by @StrikeGraph.

Website: www.securetalkpodcast.com
Youtube: https://youtu.be/eZFgxKsFvYg?si=w62KtsRa4dNGEl3b
Apple Podcast: https://podcasts.apple.com/us/podcast/predicting-data-breach-risk-how-mathematical-privacy/id1354145110?i=1000700773770
Spotify: https://open.spotify.com/episode/4o9h8PPy6jnKJzYyLqxUDv
SoundCloud: https://soundcloud.com/user-779694357/predicting-data-breach-risk-how-mathematical-privacy-is-revolutionizing-data-sharing

This article does a great job highlighting why DOGE is taking over the federal government so easily: federal systems centralize massive amounts of sensitive data, making them highly vulnerable to insider threats. The article concludes by pointing out that techniques like #FederatedLearning and #DifferentialPrivacy could help build more resilient systems 👏 #Privacy #CyberSecurity

https://www.nytimes.com/2025/02/21/opinion/musk-doge-personal-data.html?unlocked_article_code=1.yk4.ioNW.2SNQKCzmcCwR

The OpenDP project has published a Q&A with me on the occasion of the #DifferentialPrivacy book, which is launching on March 25: https://opendp.org/blog/qa-simson-garfinkel-author-upcoming-book-differential-privacy

Time for a proper #introduction! 👋 I'm a #MachineLearning researcher at Inria, France, focusing on #TrustworthyAI—especially #decentralization & #privacy. I design algorithms that learn from #decentralized data (#FederatedLearning) without memorizing personal data (#DifferentialPrivacy), working toward putting #data & #AI back in people's hands. Maybe one day, these ideas could be used in the #Fediverse!

"To prevent AI models from memorizing their input, we know exactly one robust method: differential privacy (DP). But crucially, DP requires you to precisely define what you want to protect. For example, to protect individual people, you must know which piece of data comes from which person in your dataset. If you have a dataset with identifiers, that's easy. If you want to use a humongous pile of data crawled from the open Web, that's not just hard: that's fundamentally impossible.

In practice, this means that for massive AI models, you can't really protect the massive pile of training data. This probably doesn't matter to you: chances are, you can't afford to train one from scratch anyway. But you may want to use sensitive data to fine-tune them, so they can perform better on some task. There, you may be able to use DP to mitigate the memorization risks on your sensitive data.

This still requires you to be OK with the inherent risk of the off-the-shelf LLMs, whose privacy and compliance story boils down to "everyone else is doing it, so it's probably fine?".

To avoid this last problem, and get robust protection, and probably get better results… Why not train a reasonably-sized model entirely on data that you fully understand instead?"

https://desfontain.es/blog/privacy-in-ai.html

#AI #GenerativeAI #LLMs #SLMs #Privacy #DifferentialPrivacy #Memorization

Revolutionizing Privacy in AI: RAG Meets Differential Privacy

In a groundbreaking study, Nicolas Grislain introduces a novel approach to enhance privacy in Retrieval-Augmented Generation (RAG) systems. By integrating differentially private token generation, this...

https://news.lavx.hu/article/revolutionizing-privacy-in-ai-rag-meets-differential-privacy

#news #tech #DifferentialPrivacy #RetrievalAugmentedGeneration #EthicalAI

Unlocking Privacy in Retrieval-Augmented Generation: Meet Sarus DP-RAG

In an era where data privacy is paramount, Sarus DP-RAG emerges as a groundbreaking solution that blends retrieval-augmented generation with differential privacy. This innovative approach not only enh...

https://news.lavx.hu/article/unlocking-privacy-in-retrieval-augmented-generation-meet-sarus-dp-rag

#news #tech #DifferentialPrivacy #RetrievalAugmentedGeneration #AIPrivacy

Datenschutzbedenken bei neuer Foto-Suche
Apple hat in iOS 18, iPadOS 18 und macOS Sequoia neue Funktionen für die Fotos-App eingeführt, die von vielen Nutzer:innen kritisch betrachtet werden. Die „Erweiterte visuelle Suche“, eine KI-gestützte Funktion zu
https://www.apfeltalk.de/magazin/news/datenschutzbedenken-bei-neuer-foto-suche/
#News #Services #AppleDatenschutz #Datenschutz #DifferentialPrivacy #ErweiterteVisuelleSuche #FotosApp #IOS18 #JeffJohnson #KIFunktionen #MacOSSequoia #Optin

#DifferentialPrivacy has a cover.
Paperback
ISBN: 9780262551656
Pub date: March 25, 2025
Publisher: The MIT Press
244 pp., 5 x 7 in, 22 b&w illus.

Advance copies available for reviewers! Email me at [email protected] for info.

https://mitpress.mit.edu/9780262551656/differential-privacy/

And we're off! 🛫 Lots of new faces this year in the Vis Lab @Northeastern @KhouryCollege ! We're excited to continue building community and doing meaningful research together 💙 #DataVisualization #HCI #Accessibility #XAI #ML #DataArt #Health #Astronomy #DifferentialPrivacy