Gaussian Process sampling and marginalisation

Welcome back. The goal in this tutorial video is to experiment with the notions of discrete and continuous Gaussian Processes, and also with the notions of sampling and marginalisation of Gaussian Processes. So let’s start, as we often do, by loading a statistical shape model here, that is stored, again, in this file bfm.h5, and displaying it in the scene. Now, we saw previously that a statistical mesh model is actually a wrapper around the Gaussian Process. And this is why now we can actually access this Gaussian Process and directly sample from it, as we do here, by actually accessing this GP field of the model.

And when I do this and actually display the sample in the field, you see that I get this discrete deformation field. And this you can also see in actually the return type in Scalismo. So when we sample from such a GP, we actually get back a discrete vector field. And this is now a discrete vector valued function that is defined over a discrete set of points.

And the reason why we obtain, actually, such a discrete function is because the Gaussian Process that is associated with the statistical mesh model is itself a discrete one. If we now want to sample continuous vector valued functions, which is what we’re usually interested in in statistical shape modelling, what we can do is we can actually interpolate the Gaussian Process. And here, what we do is we actually call this interpolate nearest neighbour method on the discrete Gaussian Process, which then returns us a continuous Gaussian Process. I will let you check out the details of this interpolation method in the companion document to this tutorial video.

If we now sample from call the sample method on this continues Gaussian Process that we just obtained, you see here that what we get back now is a vector field, 3D 3D, which in Scalismo is actually the class of continuous vector valued functions. So this is now a function that is defined on a continuous domain. And we can actually now evaluate it at any set of points we choose. Now that we have a continuous Gaussian Process, what we know from Gaussian Process theory is that we can actually evaluate it at any set of points we choose. So for example, here I can choose to evaluate it on all points of the mean mesh of our face model.

And this I can do by calling this sampleAtPoints method that actually takes as a parameter a discrete domain. And in Scalismo, all of the triangle meshes are valid discrete domains. And this is why we can actually feed them directly, or here I can directly feed this mean mesh to this method and obtain back discrete deformation field. And if I now evaluate this and show this field in the scene, you will see that we obtain a deformation field where actually all of the vectors are defined, or in other words, starting from points on the mean mesh.

Since we can actually choose to evaluate the Gaussian Process on any set of points, we actually might as well evaluate it on a single point. So what I’m doing here is actually calling this sampleAtPoints method again on this single point domain, and I’m constructing this single point domain using this unstructuredPointsDomain constructor where I’m passing it a list of points containing only one element that is actually this point on the mean mesh that happens to be a point on the tip of the nose of the mean face. And then what I’m doing is I’m again, using this domain, I’m sampling from the Gaussian Process, which gives me a discrete deformation field, and displaying it in this scene.

And you see here that we actually now have one sample that is consisting of a single deformation vector that is actually defined only on this tip of the nose. What we also know from Gaussian Process theory is that the second we fix the set of points on which we’re interested in evaluating our deformations to a discrete one, then we can think of the Gaussian Process as a multivariate normal distribution. So what if we are now interested in retrieving this multivariate normal distribution, not just sampling a few instances? Well, this is this concept marginalisation of a Gaussian Process. And we can apply this concept in Scalismo by calling this marginal method here on our continuous Gaussian Process.

And in a similar way to the sampleAtPoints method, the parameter that we need to pass here is again a discrete domain on which we would like to marginalise. And here, what I’m doing is I’m constructing this discrete domain using, again, this unstructuredPointsDomain constructor. And I’m building here a domain that is consisting of only two points one point on the right eye of the mean mesh, and one point on the left eye of the mean mesh. And this is how I actually retrieve the position of these two points. If we now look at the return type of this marginal method, you see that we get back, actually, a discreet Gaussian Process. So this is now a full distribution over deformation fields.

And this is to contrast with the sample at points that returned as only one discrete vector field as an instance or as a sample from this model.

So now that I have this distribution of our deformation field, what I can do is, again, I can now sample and display the types or the sample deformations in my scene, and this is what I do here.

And you see now that the samples of the marginal or the deformation fields that are sampled from this marginal Gaussian Process are now discrete vector fields that are defined only over these two points. And now every time that I call this sample method again, since I have now a normal distribution, I get back a different sample. And you see that in some cases, actually, the vector ends up going sometimes inside the eyes as well.

So as we saw previously, a statistical mesh model is just a wrapper around a discrete Gaussian Process, and this is why this concept of marginalisation of Gaussian Processes also extends to this class of statistical mesh models in Scalismo. So in a similar way, we can actually call this marginal method directly on this face model that we just loaded. And the only difference here is that instead of passing a discrete domain in this case, we actually pass, if you notice here, a list or an index sequence of point identifiers that we just computed here above. And the reason for doing this is that the Gaussian Process that is associated with our face model is actually already a discrete Gaussian Process.

And marginalising a discrete Gaussian Process in Scalismo is actually selecting a subset only of the discrete set of points on which it is already defined. And so here, I executive now this part of the code here where we actually start by retrieving the identifiers of the points that are on the nose of the mean mesh, and then marginalising over these points, and finally, showing our model in the scene. And as you can see here now, we actually obtain a statistical mesh model, or a marginal model, that is defined only on these points, and actually gives us deformations only of these points. And now with this marginalisation, we actually have a statistical nose model.

So I encourage you now to go through the companion document to this tutorial and give these parts of codes a try.