Skip to 0 minutes and 7 seconds Welcome to Week 3. In this week, we’re going to explore the probabilistic aspects of shape modelling. We start in this video by discussing the marginalisation property. The marginalisation property of a Gaussian Process is the property that makes it not only a nice theoretical concept but actually a very practical tool that we can use for shape modelling.

Skip to 0 minutes and 36 seconds We started exploring Gaussian Processes by modelling discretely-defined deformation fields using the concept of a multivariate normal distribution.

Skip to 0 minutes and 50 seconds We then introduced a mean function and a covariance function which allowed us to generalise the concept of multivariate normal distribution through the concept of a Gaussian Process. The Gaussian Process provides us with a continuous representation.

Skip to 1 minute and 8 seconds From a conceptual point of view, we like it much better to work with the Gaussian Process. Somehow, it provides us a simple and compact definition in terms of just a mean function and a covariance function, which is nice to work with. It is more powerful. So we could theoretically model infinite dimensional distribution. But the most important point is actually that a Gaussian Process makes us independent of the discretisation of the shape.

Skip to 1 minute and 45 seconds Now, there is a question that we have to answer here. Because we’re not only interested in mathematical beauty, but at the end of the day, we want to use shape modelling in practical applications which are implemented on a computer. Now, the representation as a multivariate normal distribution which is discrete is much better suited for such an implementation.

Skip to 2 minutes and 11 seconds Luckily, Gaussian Processes give us both the mathematical beauty plus a way to go to this discrete representation such that they become useful and practised.

Skip to 2 minutes and 26 seconds This is thanks to a very special property of the multivariate normal distribution, which is called the ‘marginalisation property.’ Let us assume that we have two sets of random variable, a set x and a set y. And we model them using a joint multivariate normal distribution.

Skip to 2 minutes and 49 seconds Now, sometimes, we’re interested in only the distribution of a set of variables let’s say in the set of variables x. This is called the ‘marginal distribution’. For general distributions, to get the marginal distribution, we would have to integrate out all the variables we’re not interested in, which is often not possible. But when we have a multivariate normal distribution, this operation becomes really simple. Namely, we know that the marginal distribution is again a multivariate normal distribution and the mean and the covariance, they are just the corresponding blocks that we had in the joint distribution. So essentially, what we’re doing is we’re disregarding all the information that involves why.

Skip to 3 minutes and 47 seconds Now, this property can even be used to formally define a Gaussian Process, and it’s easy to believe that the same property kind of holds for a Gaussian Process because a Gaussian Process is just a generalisation of a multivariate normal distribution.

Skip to 4 minutes and 8 seconds We can formally define a Gaussian Process as a probability distribution over functions which has the property that, whenever we look only at the finite number of values here, x1 to xn then the distribution of these function values is a multivariate normal distribution. And the mean and the covariance is just obtained by evaluating the mean function on these points, x1 to xn. And the covariance matrix is obtained by evaluating the covariance function and all pairs of points. What this implies is that, if we define the Gaussian Process on a continuously-defined set of points, we can always go to a discrete representation by restricting the attention to a finite number of points.

Skip to 5 minutes and 6 seconds For example, if you’re only interested in getting the distribution for a set of landmark points, that’s no problem. We can just do that, and we know that the distribution is a multivariate normal distribution where we have the mean and the covariance just obtained by evaluating the corresponding function at these points. Sometimes, we’re also interested in a dense representation, and it’s just the same. We just get the corresponding mean and covariance matrix directly from the process. And even sometimes, we just want to restrict the model to a part of the shape let’s say the thumb. Also, this is no problem. We just get a corresponding multivariate normal distribution which gives us all the information about how these points are distributed.

Skip to 6 minutes and 3 seconds Since it’s relatively simple to work with multivariate normal distribution and we know a lot about this type of distribution, we can also do interesting things with it. So for example, we can easily sample from a multivariate normal distribution.

Skip to 6 minutes and 23 seconds This allows us to visualise shape variation. So what we would do is we would say what points we want to obtain samples from, we would compute the multivariate normal distribution, and then sample from that. And this gives us an impression, how do the deformations look that come from this shape model.

Skip to 6 minutes and 44 seconds We can also use the multivariate normal distribution to compute probabilities. We can answer a question like ‘what is a normal variation’, or ‘is a given variation still normal?’ So for example, if we have the marginal distribution around this point x, then we can think that, if we ever observe a hand where the tip of the thumb actually goes out of the shaded area, then we know that it is improbable that this is a valid hand shape. And we can do that for actually many more points we can compute the marginal distribution around every point if I want. And this gives us something like a confidence region, which tells me something about the variance around each point.

Skip to 7 minutes and 36 seconds And I would know that, if a hand actually falls out of this grey confidence region, then it would be a very unlikely shape.

Skip to 7 minutes and 48 seconds I hope that you will be exploring more of these probabilistic concepts in the exercises using Scalismo Lab.

# The marginalisation property

Gaussian Processes provide us with a mathematically elegant way of modelling shape deformations. As shape modelling is an application-oriented task, we are not primarily interested in mathematical elegance, but rather in obtaining practical algorithms.

Thanks to the *marginalisation property*, Gaussian Processes also satisfy this criterion. We will discuss how the marginalisation property allows us to obtain a discrete representation of the shape variation in terms of a multivariate normal distribution and how we can use this to obtain interesting information about the shapes that are represented by the model.

© University of Basel