#spmathsblog — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #spmathsblog, aggregated by home.social.
-
The optimal constant in Hölder-Rogers-type uniform continuity assumption satisfies an interesting condition with respect to its index parameter, with some superficial connections to exponential family theory. In this post, I briefly describe this result and its proof.
'Log-Convexity of the \( C^\alpha \) Semi-Norm'
https://hackmd.io/@sp-monte-carlo/S1IAwHtHn -
Certain methods and concepts become clearest in the context of other ideas. One interesting version of this is when some new idea can be viewed as an approximation to some other thing, which was perhaps overlooked or viewed as out-of-reach. This post briefly discusses some aspects of this phenomenon.
'The “Approximation to What?” Principle'
https://hackmd.io/@sp-monte-carlo/SkWUiBFHn -
A condensation of a few perspectives on the rising topic of diffusion models, focusing less on my usual areas of comfort (i.e. Markov processes and friends) and more on the other practical core, (i.e. denoising operators).
'Denoising-Centric Diffusions'
https://hackmd.io/@sp-monte-carlo/SJWAZBtr2 -
Various sampling algorithms can (implicitly or explicitly) involve some recursive structure, which enables certain tools for their theoretical analysis. Here is a small write-up of some basic thoughts on the topic.
https://hackmd.io/@sp-monte-carlo/BkHYt7tBn
'Nested Structure in MCMC Algorithms' -
A small note on a reinterpretation / strengthening of a classical probabilistic inequality.
'Hoeffding’s Inequality by Convex Ordering'
-
A proper write-up of the ramblings above, for easier reference: (I can't remember how much, if at all, I edited things)
'Logarithmic Sobolev Inequalities and Logarithmic Lipschitz Regularity'
-
Adding some details to an earlier post (https://mathstodon.xyz/@sp_monte_carlo/109542712262310989): deriving and describing some near-Gaussian distributions which are relatively { explicit, tractable } in some (mildly) useful ways:
'Isoperimetric Surrogates for the Gaussian'
-
Taking stock of the past week (with apologies for the dense formatting).
'#BayesComp2023 Recap'
-
Earlier this year, I learned one of my current favourite 'concise results', i.e. results where you can concisely communicate the substance of the result and why it is true. In this post, I offer a snappy presentation to this effect.
'Univariate Log-Concave Rejection Sampling is Solved'
-
So-called 'implicit' statistical models raise interesting opportunities for their ability to accommodate complex mechanistic data-generating processes, while posing a number of computational challenges. Many approaches to inference in these settings involves the systematic formation and refinement of approximations to both the likelihood and posterior. In this post, I outline some abstract descriptions of this approach.
'Iterative Likelihood Approximation'
-
One of the few things which all Markov chains have in common with one another is the use of a time variable. As such, one can probe the structure of general Markov chains by examining how they look both forwards and backwards in time. This post collects some very brief thoughts on the matter.
'Forwards, Backwards, and Stochastic Formulations of Optimal Transport'
-
Control variates are a useful tool for variance reduction in various Monte Carlo contexts. However, depending on the context, the methodology for fitting control variates ought to change slightly; I describe some of the changes here.
'Fitting Control Variates in MCMC'
-
There are various different streams of Monte Carlo methods, and it is sometimes tempting to frame them as being in competition with one another (RS vs IS, MCMC vs SMC, etc.). In this short post (prompted by a relatively old tweet of mine, from which my views have since shifted slightly), I argue against this framing, and advocate for celebrating the synergies and similarities between the sub-fields.
'Competition between Monte Carlo Methods'
-
In analysis, one often wants to compare two quantities which arise as the outcome of time-varying processes, perhaps different ones. There are some nice tools which allow for these comparisons to be made, particularly for processes with good stability properties. The Alekseev-Groebner decomposition is one such tool.
'Decompositions à la Alekseev-Groebner'
-
Two recurrent difficult scenarios in probabilistic computations are dealing with multimodality and heavy-tailedness. Multimodality is difficult, but comes with some reasonable 'default' solutions, whereas heavy tails are perhaps less damaging, but require more care to resolve. In this blog post, I expand upon this distinction somewhat.
'Computation of Heavy-Tailed Measures'
-
It is well-understood that many sampling algorithms can mix more rapidly when some of the components of the target measure are marginalised out. In this blog post, I provide a quick proof of this ordering in a specific setting.
'On the Benefits of Marginalisation for Langevin Diffusions'
-
A short note on a technique for remembering how to compute the Schur complement of a PSD matrix, which can arise in computing conditional laws of Gaussian random variables (which I think was the motivation for writing this).
'A Mnemonic for Schur Complements'
-
Exponential families are often held up as being a very nice collection of probability distributions with which to work, particularly in a statistical context. This characterisation is very frequently accurate in practice, but there can be more to the story than just the presence of exponential families:
'Convenient Exponential Families'
-
Many tasks require demonstrating that a system is eventually exponentially convergent to an equilibrium state. In this post, I describe how statements of this form can be systematically improved to show uniform exponential convergence to equilibrium for a suitable related function.
'Contractivity from Eventual Exponential Convergence'
-
Some notes on the challenge of diagnosing convergence for MCMC algorithms, and how much of the algorithms' structure should be brought into this task.
'Convergence Diagnostics for MCMC'
-
A fun puzzle based on the idea of replacing the squared displacement loss in optimal transport with a Bregman divergence, and how this modification impacts the structure of optimal couplings:
'Bregman Cost Functions in Optimal Transport'
-
Cobbling together some thoughts about transport maps, optimal and not:
'Non-Optimal Transport Maps'
-
Some notes on some more 'quiet assumptions' in MCMC, this time with a focus on Gibbs sampling:
'Scans in Gibbs Sampling'
-
Over the past few years, I have gotten a real taste for reformulating various discrete-time stochastic algorithms as continuous-time jump processes, which often furnishes them with nice properties. Here is a less well-known example:
'Multiple-Try Metropolis via Jump Processes'
-
Closing out the calendar year by revisiting some recurrent aspects of MCMC algorithms which may or may not be necessary:
'Some Quiet Assumptions in MCMC'
-
I have written up some details for a problem about which I was curious a couple of months ago. The topic: understanding whether the sets of
[ convex functions diverging at infinity ]
and
[ square roots of strongly-convex functions ]
might be 'the same', up to some small errors and conditions.
'Strong Convexity and Square Roots'
-
Another old thread of posts comes in for the 'light editing and reformatting' treatment here. It's short and very informal, and mostly just consists of me trying to distill a few of my own thoughts on the matter.
'Principles of the Monte Carlo Method'
-
I had a bullet point in a notes file from some time ago, as a reminder to write something bloggy on this topic. It's a tricky area for me: the high-level questions seem extremely difficult at a fundamental level, and the ground-level questions are not so well-aligned to my own expertise. Still, it was nice to write a few things which I've been able to abstract { and / or } understand a bit.
'Minibatch Markov Chain Monte Carlo'
-
A small-ish post here, concerning a relatively self-contained puzzle in { probability / estimation }.
https://hackmd.io/@sp-monte-carlo/HyebhqgYs
'Unbiased Estimators of the Inverse of the Mean from a Poisson Stream'
-
Decided to expand another thread into a blog post, while the topic was still relatively fresh on my mind:
https://hackmd.io/@sp-monte-carlo/SyxUoeQJto
'Why not Orlicz Norms?'