#gaussianprocess — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #gaussianprocess, aggregated by home.social.
-
This blog post about creating Gaussian processes from scratch in Python helped me to make sense it. https://peterroelants.github.io/posts/gaussian-process-tutorial/
2/n
-
This blog post about creating Gaussian processes from scratch in Python helped me to make sense it. https://peterroelants.github.io/posts/gaussian-process-tutorial/
2/n
-
This blog post about creating Gaussian processes from scratch in Python helped me to make sense it. https://peterroelants.github.io/posts/gaussian-process-tutorial/
2/n
-
This blog post about creating Gaussian processes from scratch in Python helped me to make sense it. https://peterroelants.github.io/posts/gaussian-process-tutorial/
2/n
-
This blog post about creating Gaussian processes from scratch in Python helped me to make sense it. https://peterroelants.github.io/posts/gaussian-process-tutorial/
2/n
-
A Gaussian process is a random function, which for me was hard to understand. How do you get these smooth functions if the value of the function at each x position is random?
Now it's making more sense to me. You can pull the whole function out of a hat! The distribution has correlations that encode relationships between the different dimensions (x values). So even though the whole function is random, adjacent points on the function could be closely related.
-
A Gaussian process is a random function, which for me was hard to understand. How do you get these smooth functions if the value of the function at each x position is random?
Now it's making more sense to me. You can pull the whole function out of a hat! The distribution has correlations that encode relationships between the different dimensions (x values). So even though the whole function is random, adjacent points on the function could be closely related.
-
A Gaussian process is a random function, which for me was hard to understand. How do you get these smooth functions if the value of the function at each x position is random?
Now it's making more sense to me. You can pull the whole function out of a hat! The distribution has correlations that encode relationships between the different dimensions (x values). So even though the whole function is random, adjacent points on the function could be closely related.
-
A Gaussian process is a random function, which for me was hard to understand. How do you get these smooth functions if the value of the function at each x position is random?
Now it's making more sense to me. You can pull the whole function out of a hat! The distribution has correlations that encode relationships between the different dimensions (x values). So even though the whole function is random, adjacent points on the function could be closely related.
-
A Gaussian process is a random function, which for me was hard to understand. How do you get these smooth functions if the value of the function at each x position is random?
Now it's making more sense to me. You can pull the whole function out of a hat! The distribution has correlations that encode relationships between the different dimensions (x values). So even though the whole function is random, adjacent points on the function could be closely related.
-
I’m excited to announce our #SIGGRAPHAsia2025 paper, which makes #rendering of #GaussianProcess implicit surfaces (GPISes) practical:
Project page: https://cs.dartmouth.edu/~wjarosz/publications/xu25practical.html
We achieve this with a novel #procedural noise formulation and by enabling next-event estimation for specular BRDFs. [1/7]
-
I’m excited to announce our #SIGGRAPHAsia2025 paper, which makes #rendering of #GaussianProcess implicit surfaces (GPISes) practical:
Project page: https://cs.dartmouth.edu/~wjarosz/publications/xu25practical.html
We achieve this with a novel #procedural noise formulation and by enabling next-event estimation for specular BRDFs. [1/7]
-
I’m excited to announce our #SIGGRAPHAsia2025 paper, which makes #rendering of #GaussianProcess implicit surfaces (GPISes) practical:
Project page: https://cs.dartmouth.edu/~wjarosz/publications/xu25practical.html
We achieve this with a novel #procedural noise formulation and by enabling next-event estimation for specular BRDFs. [1/7]
-
I’m excited to announce our #SIGGRAPHAsia2025 paper, which makes #rendering of #GaussianProcess implicit surfaces (GPISes) practical:
Project page: https://cs.dartmouth.edu/~wjarosz/publications/xu25practical.html
We achieve this with a novel #procedural noise formulation and by enabling next-event estimation for specular BRDFs. [1/7]
-
I’m excited to announce our #SIGGRAPHAsia2025 paper, which makes #rendering of #GaussianProcess implicit surfaces (GPISes) practical:
Project page: https://cs.dartmouth.edu/~wjarosz/publications/xu25practical.html
We achieve this with a novel #procedural noise formulation and by enabling next-event estimation for specular BRDFs. [1/7]
-
Open Ocean #7
Previously on Open Ocean. I did some pattern estimation using an iterative algorithm that worked like magic. But before that, I made a simple Gaussian Process interpolator.
One thing we can use the Gaussian Process interpolation for is to estimate biases in the ship data at a global and ship-by-ship scale. The process is to first grid up the ship and drifter data separately and then sequentially assimilate the buoy and ship observations. The drifter data provides an initial unbiased estimate of the global field. The ship data are then assimilated and the resulting field is effectively bias adjusted.
Ship data
First I grid up the ship data. I’m using July 2008 because those are the random numbers my fingers hit. I add a constant term to the error covariance to represent a global bias common to all ships. The uncertainty associated with this is set to 0.2K, which is larger than the bias is likely to be but is roughly the right order of magnitude.
The map of gridded anomalies from ships shows some interesting features. There is a short El Nino warm tongue in the eastern Pacific. The North Atlantic looks to be largely warmer than the average. There is a strong warm anomaly in the western North Pacific with cooler anomalies in the east. In the southeast Pacific there is a lonely ship track with a very positive anomaly. This is likely erroneous.
Drifter data
I gridded the drifter data separately, but in this case I haven’t added a constant to the error covariance reflecting the fact (my belief anyway) that drifters are unbiased. If you’re worried about that, we can take an operational view and just say that the “true” SST is whatever drifting buoys measure on average. The error covariances still allow for biases in individual buoys, but the mean bias is assumed to be zero.
While the coverage of drifting buoy observations is more complete than for ships, there are still significant gaps, including in the tropical Pacific. This is why it can be important to use ship data despite the fact that it is less reliable than drifter data: it still contains useful information. In this case, ships provide information which will help to reconstruct the El Nino. On the other hand, the drifter anomalies in the Atlantic look to be cooler, on average, than the ship data so the drifters will help there (and elsewhere).
Interpolation step 1
I set up the Gaussian Process using the default HadCRUT5 parameters and then interpolate the drifter data. As with the ship data, I add a constant term to the prior covariance for the reconstruction, which represents a “global mean” temperature. Once the interpolation is done, I mask the land areas out for plotting. The first step interpolation looks like this.
The interpolated drifter data have most of the same features as the basic gridded drifter data. They’re just a bit smoother and there are no gaps except where there’s land.
Interpolation step 2 and bias adjustment
Next, I subtract the first interpolation from the ship data. I then use the posterior distribution from the first step of interpolation as a prior for the second step. I then do the interpolation of the ship data and add the interpolated ship data back onto the first interpolation step.
The SST anomalies from the second interpolation step aren’t that different from the first suggesting that the ship data are not adding a great deal of information. However, they are providing some information in the tropical Pacific. There are other minor changes too (we’ll come back to the Arctic).
Estimated ship errors
In principle, we can explicitly estimate each component of the ship errors: large scale bias, ship by ship bias, measurement error and sampling error. Each one is just a component of the overall variance. Mathematically, their covariance matrices are no different from the covariances describing the actual SST field, so, just as we can estimate the SST field using this technique, we can estimate the errors. However, I haven’t done that. I’ve just taken the difference between the gridded ship data and the second step reconstruction. It’s shown below.
First off, you can see the pervasive warm bias. There’s noise because a lot of the error terms are noisy. On the other hand, the ship track joining the Panama Canal to Australia and New Zealand shows up as a swath of positive biases. To the south of that, as expected, the solitary ship in the southeast Pacific has a clear warm bias, but you can also see a cold biased ship which track from the southern tip of South America, slightly west of North up to Central America.
A similar result can be had by just taking the difference between the gridded ship and drifter data. However, the field is less complete, noisier because it includes noise from the drifter data too, and therefore we’re not making full use of the available information.
Doing it properly
OK, so, I calculated the bias components of the ship error on their own. It looks like this ⬇️. I narrowed the colour scale (it now runs from -1 to +1 rather than -3 to +3) to better show off the features. With the noise from uncorrelated measurement and sampling errors removed, you can clearly see the ship tracks criss-crossing the various ocean basins. The pale pink background is associated with the constant term in the error covariance and represents the “global ship bias”.
This still doesn’t quite have the rich spectrum of biases that exists in the real world. We know, for example, that errors correlate at the level of individual data collections (in ICOADS these were called “decks” meaning decks of punched cards on which the data were stored) or country of registration. I don’t have that information in IQUAM, so I can’t deal with that at the moment. Also, SST biases can be subdivided into biases associated with different measurement methods, but IQUAM doesn’t have that metadata either. Even within one measurement method there might be different kinds of bias behaviour. Some ships making measurements in their engine rooms have biases that are anticorrelated with the SST (the bias is bigger when the water is colder). It’s easy enough to add these terms to the covariance if the appropriate metadata can be found.
Bias adjusted time series
By repeating the 2-step interpolation for every month it’s possible to build up a time series that integrates information from the drifting buoys and ships in some kind of “optimal” way1. One nice outcome of this is that the data are bias adjusted in the process.
The interpolated data uses both ship and drifter data, but unlike the simple gridded combination of the two, the series now tends to follow the drifting buoy data throughout. Gridding them together without compensating for the bias in the ship data, leads to an artificial cooling in the combined dataset, but that should now be gone.
Of course, we might have other problems instead. In the 1980s, the coverage of the drifting buoys is very far from global and those buoys that there are take fewer and noisier measurements. Ideally, we would use a different benchmark dataset to continue adjusting the data prior to the drifter era (ERSST uses marine air temperatures, HadSST4 uses oceanographic measurements). Also, I haven’t yet included the moored buoy data in the interpolation and the Arctic Ocean has an unrealistically warm anomaly so I will need to do something about ice. There’s ice up there which moderates the anomalies by pinning them close to whatever the freezing point of sea water happens to be.
Comparison to HadSST4
We can compare this series to that from HadSST4. Previously, I compared the combined series to the unadjusted HadSST4, but here I’m going to compare to the adjusted version because we should have a series that is effectively homogenised. So, let’s see…
That’s surprisingly close, I would say.
There are a couple of things to note. First, the coverage of the two data sets is different because HadSST4 is not interpolated so we might still expect some differences. Second, prior to 1995 or thereabouts, the two diverge. That’s because HadSST4 uses subsurface data in addition to drifter data to benchmark the bias adjustments. In principle, HadSST4 is doing a better job. Third, the two are still referenced to different baselines. I’ve removed the average offset here, but not accounted for any seasonal effects.
For all that though, and despite being less sophisticated, this simple series looks to be doing quite a good job (since the mid 1990s anyway).
-fin-
- I put the prophylactic bunny ears on “optimal” because the whole thing requires fine tuning and it’s only optimal in the sense that we got everything right, which is to say it’s actually suboptimal. ↩︎
#climate #climateChange #gaussianProcess #interpolation #observations #seaSurfaceTemperature
-
We propose a new family of probability densities that have closed form normalising constants. Our densities use two layer neural networks as parameters, and strictly generalise exponential families. We show that the squared norm can be integrated in closed form, resulting in the normalizing constant. We call the densities Squared Neural Family (#SNEFY), which are closed under conditioning.
Accepted at #NeurIPS2023. #MachineLearning #Bayesian #GaussianProcess
-
We propose a new family of probability densities that have closed form normalising constants. Our densities use two layer neural networks as parameters, and strictly generalise exponential families. We show that the squared norm can be integrated in closed form, resulting in the normalizing constant. We call the densities Squared Neural Family (#SNEFY), which are closed under conditioning.
Accepted at #NeurIPS2023. #MachineLearning #Bayesian #GaussianProcess
-
We propose a new family of probability densities that have closed form normalising constants. Our densities use two layer neural networks as parameters, and strictly generalise exponential families. We show that the squared norm can be integrated in closed form, resulting in the normalizing constant. We call the densities Squared Neural Family (#SNEFY), which are closed under conditioning.
Accepted at #NeurIPS2023. #MachineLearning #Bayesian #GaussianProcess
-
We propose a new family of probability densities that have closed form normalising constants. Our densities use two layer neural networks as parameters, and strictly generalise exponential families. We show that the squared norm can be integrated in closed form, resulting in the normalizing constant. We call the densities Squared Neural Family (#SNEFY), which are closed under conditioning.
Accepted at #NeurIPS2023. #MachineLearning #Bayesian #GaussianProcess
-
We propose a new family of probability densities that have closed form normalising constants. Our densities use two layer neural networks as parameters, and strictly generalise exponential families. We show that the squared norm can be integrated in closed form, resulting in the normalizing constant. We call the densities Squared Neural Family (#SNEFY), which are closed under conditioning.
Accepted at #NeurIPS2023. #MachineLearning #Bayesian #GaussianProcess
-
"Neural network with optimal neuron activation functions based on additive Gaussian process regression"
https://arxiv.org/abs/2301.05567 -
"Neural network with optimal neuron activation functions based on additive Gaussian process regression"
https://arxiv.org/abs/2301.05567 -
"Neural network with optimal neuron activation functions based on additive Gaussian process regression"
https://arxiv.org/abs/2301.05567 -
"Neural network with optimal neuron activation functions based on additive Gaussian process regression"
https://arxiv.org/abs/2301.05567 -
"Neural network with optimal neuron activation functions based on additive Gaussian process regression"
https://arxiv.org/abs/2301.05567 -
"Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic Systems"
https://arxiv.org/abs/2208.06528#DynamicalSystems #ModelCalibration #Bayesian #ParameterEstimation #GaussianProcess
-
"Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic Systems"
https://arxiv.org/abs/2208.06528#DynamicalSystems #ModelCalibration #Bayesian #ParameterEstimation #GaussianProcess
-
"Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic Systems"
https://arxiv.org/abs/2208.06528#DynamicalSystems #ModelCalibration #Bayesian #ParameterEstimation #GaussianProcess
-
"Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic Systems"
https://arxiv.org/abs/2208.06528#DynamicalSystems #ModelCalibration #Bayesian #ParameterEstimation #GaussianProcess
-
"Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic Systems"
https://arxiv.org/abs/2208.06528#DynamicalSystems #ModelCalibration #Bayesian #ParameterEstimation #GaussianProcess
-
Take the idea of random Fourier features, as applied to #GaussianProcess regression in #MachineLearning. There is a method in the probabilistic numerics textbook about Gaussian quadrature (same Gauss, different method) which gives good convergence with respect to the spectrum of a function. Show that a high quality #kernel (low rank approximation) can be computed efficiently (sublinear in the number of training points).
https://www.jmlr.org/papers/v23/21-0030.html -
Take the idea of random Fourier features, as applied to #GaussianProcess regression in #MachineLearning. There is a method in the probabilistic numerics textbook about Gaussian quadrature (same Gauss, different method) which gives good convergence with respect to the spectrum of a function. Show that a high quality #kernel (low rank approximation) can be computed efficiently (sublinear in the number of training points).
https://www.jmlr.org/papers/v23/21-0030.html -
Take the idea of random Fourier features, as applied to #GaussianProcess regression in #MachineLearning. There is a method in the probabilistic numerics textbook about Gaussian quadrature (same Gauss, different method) which gives good convergence with respect to the spectrum of a function. Show that a high quality #kernel (low rank approximation) can be computed efficiently (sublinear in the number of training points).
https://www.jmlr.org/papers/v23/21-0030.html -
Take the idea of random Fourier features, as applied to #GaussianProcess regression in #MachineLearning. There is a method in the probabilistic numerics textbook about Gaussian quadrature (same Gauss, different method) which gives good convergence with respect to the spectrum of a function. Show that a high quality #kernel (low rank approximation) can be computed efficiently (sublinear in the number of training points).
https://www.jmlr.org/papers/v23/21-0030.html -
Take the idea of random Fourier features, as applied to #GaussianProcess regression in #MachineLearning. There is a method in the probabilistic numerics textbook about Gaussian quadrature (same Gauss, different method) which gives good convergence with respect to the spectrum of a function. Show that a high quality #kernel (low rank approximation) can be computed efficiently (sublinear in the number of training points).
https://www.jmlr.org/papers/v23/21-0030.html -
@FCAI Update on Zheyang’s status: The opponent got there on the last minute, gave an enlightening view of the position of the thesis in #BayesianModeling and #GaussianProcess es, and asked a set of broad and challenging questions. Zheyand did outstandingly well, with still some unsolved questions which he will be eager to pursue when on the job market at some point. Big congrats Zheyang Shen and many thanks opponent Chris Oates! #PhD #AaltoUniversity @FCAI
-
@FCAI Update on Zheyang’s status: The opponent got there on the last minute, gave an enlightening view of the position of the thesis in #BayesianModeling and #GaussianProcess es, and asked a set of broad and challenging questions. Zheyand did outstandingly well, with still some unsolved questions which he will be eager to pursue when on the job market at some point. Big congrats Zheyang Shen and many thanks opponent Chris Oates! #PhD #AaltoUniversity @FCAI
-
@FCAI Update on Zheyang’s status: The opponent got there on the last minute, gave an enlightening view of the position of the thesis in #BayesianModeling and #GaussianProcess es, and asked a set of broad and challenging questions. Zheyand did outstandingly well, with still some unsolved questions which he will be eager to pursue when on the job market at some point. Big congrats Zheyang Shen and many thanks opponent Chris Oates! #PhD #AaltoUniversity @FCAI
-
"Towards Improved Learning in Gaussian Processes: The Best of Two Worlds"
https://arxiv.org/abs/2211.06260#inference #GaussianProcess #ExpectationPropagation #VariationalInference #classification
-
"Towards Improved Learning in Gaussian Processes: The Best of Two Worlds"
https://arxiv.org/abs/2211.06260#inference #GaussianProcess #ExpectationPropagation #VariationalInference #classification
-
"Towards Improved Learning in Gaussian Processes: The Best of Two Worlds"
https://arxiv.org/abs/2211.06260#inference #GaussianProcess #ExpectationPropagation #VariationalInference #classification
-
"Towards Improved Learning in Gaussian Processes: The Best of Two Worlds"
https://arxiv.org/abs/2211.06260#inference #GaussianProcess #ExpectationPropagation #VariationalInference #classification
-
"Towards Improved Learning in Gaussian Processes: The Best of Two Worlds"
https://arxiv.org/abs/2211.06260#inference #GaussianProcess #ExpectationPropagation #VariationalInference #classification
-
Anyone know of an approach to construct a #GaussianProcess prior over strictly monotonic functions? #MachineLearning #Bayesian
-
Anyone know of an approach to construct a #GaussianProcess prior over strictly monotonic functions? #MachineLearning #Bayesian
-
Anyone know of an approach to construct a #GaussianProcess prior over strictly monotonic functions? #MachineLearning #Bayesian
-
Anyone know of an approach to construct a #GaussianProcess prior over strictly monotonic functions? #MachineLearning #Bayesian
-
I have a variety of #tutorials posted on my website (https://peter-stewart.github.io/posts/) on a variety of topics, including a series on classic ecological #models in #Stan and #RStats, an #occupancy model which handles spatial autocorrelation using a #GaussianProcess, and a variety of useful Windows batch files for dealing with #CameraTrap images.
Here is a quick #thread 🧵 with links to the individual posts:
1/n
-
I have a variety of #tutorials posted on my website (https://peter-stewart.github.io/posts/) on a variety of topics, including a series on classic ecological #models in #Stan and #RStats, an #occupancy model which handles spatial autocorrelation using a #GaussianProcess, and a variety of useful Windows batch files for dealing with #CameraTrap images.
Here is a quick #thread 🧵 with links to the individual posts:
1/n