Gaussian Processes and point distribution models

Welcome back. The goal in this tutorial video is to understand the relationship between point distribution models and Gaussian Processes, or statistical mesh models, as they are known in Scalismo. And also understand what happens when we actually sample from such a statistical mesh model. So let’s go ahead and read a statistical face model as we did so far. So here I’m just reading the content of this bfm.h5 file and then displaying it in the scene. And you see here that we loaded our face model and displayed it. And we can now, as we did previously, sample random instances via the user interface by clicking here on the Random button. What we can also do is also sample programmatically.

And this we can do using the sample method on the face model. And you see that when we do that, we actually get back a triangle mesh that we then display in our scene. And let’s see the result of that.

So you see here now we got a static mesh, or static instance, out of our model. Since the model is also a probability distribution and also a normal distribution, what we can do is also check the mean of this distribution. And this we can do using the mean method here that we call on the face model. And you see that this also returns us a triangle mesh that is the mean mesh of our model that we then display.

And if I now make the sample invisible, you see that we get the mean face that we usually know when loading the model.

Now, the question is, what happens when we do that? How do we actually manage to sample these triangle meshes? And how does this relate to Gaussian Processes? Well, the answer is that a statistical mesh model in Scalismo is just actually a wrapper around a Gaussian Process. So– and it actually ships with the Gaussian Process. And this Gaussian Process, or associated Gaussian Process, can be accessed by using this GP field of the face model. And here I’m allocating, or allocating this GP field to a variable called face GP. And you see that the type of this variable is actually a discrete low rank Gaussian Process. Now, we will see in the coming weeks what exactly this low rank part means.

But for now, all that matters is that the returned type is actually a Gaussian Process. And we know that such a Gaussian Process in Scalismo is actually a distribution, or normal distribution, over deformation fields. So what we can do then is actually check the mean of this distribution. So let’s do this by calling actually the mean on this now Gaussian Process.

And you see here that what we get back is actually a discrete vector field. That is a discrete deformation field in Scalismo. And we can now visualise this returned field by calling the show method on it and specifying the name for it also in the scene. And you see here now that we can visualise our mean deformation field. And if we now zoom in on, somehow, on this vector field, you can notice here that all the arrows, or all the tips of the vectors, are actually ending on points of the mean face. All of the vectors of the mean deformation fields of the Gaussian Process associated with our model actually end on the mean mesh.

Now, to see where they start from we can now load this reference mesh, the reference measure associated with our statistical mesh model. And this also can be accessed by calling this property or field reference mesh on the face model. And this is also a fixed, or a static triangle mesh. And we do this and also show it now in the scene. And if I now– so now this reference is now displayed here. If I now make it slightly transparent, you will see that all the vectors are actually starting from points on the reference mesh and then ending on points of the mean. So this is now the answer.

And this is why I said previously that a statistical mesh model is actually a wrapper around a Gaussian Process. So what really happens when we actually get the mean of the statistical mesh model is we’re getting the mean of the Gaussian Process and then actually warping the reference mesh with this deformation field, such that we’re moving every point of the reference with its corresponding deformation here, such that we can construct the mean face. And this is also what happens when we’re sampling. When we sample from a statistical mesh model, we sample a random deformation field. And then we warp the reference with this field to get the random sample.

So I now encourage you to go through the companion tutorial document to this video and give this a try for yourself.