Statistical Shape Modelling: Useful Learning Resources

In this article, we want to show you some resources that can help you to deepen your understanding of shape modelling as well as to put to practical use the things you have learned.

Gaussian Processes

We used Gaussian Processes as our fundamental modelling technique but could only show you a small part of what Gaussian Processes can offer. You can find a wealth of literature about Gaussian Processes in the field of statistics. In this course, we have used Gaussian Processes to represent spatial data. This type of Gaussian Processes is usually referred to as Gaussian random field in the statistics literature. An introduction to Gaussian random fields, with additional pointers, is given in this review article:

Abrahamsen, Petter. A review of Gaussian random fields and correlation functions, 1997.

Another area where Gaussian Processes proved to be very useful is machine learning. An excellent introduction to Gaussian Processes and their use in machine learning is given in this free online book:

Rasmussen, Carl Edward and Williams, Christopher K. I. Gaussian processes for machine learning, 2006.

If you would like to acquire an understanding of the mathematical structures associated with Gaussian Processes and their connection to kernel methods and differential equations, we strongly recommend the following article:

Steinke, Florian and Schölkopf, Bernhard. Kernels, regularization and differential equations, 2008.

Shape modelling

The approach to shape modelling that we have presented in this course is summarised in our paper

Lüthi, Marcel, et al. Gaussian process morphable models, 2017.

We have applied our approach to many modelling problems. The most complex example is the construction of the Basel Face Model. We have open sourced the complete model construction pipeline as well as our code for analysing 2D face images. This pipeline, together with the accompanying paper, should serve as a good example of how the methods you learned in this course can be applied to challenging modelling problems.

Our approach is by no means the only possible way to model shapes. There are many other approaches, which differ in the way the shapes are represented or use different mathematical tools for modelling.

Many modern shape modelling methods are described in this book:

Zheng, G. et al. Statistical Shape and Deformation Analysis, 2017.

The excellent review article by Tobias Heimann et al. also provides a good summary of shape modelling methods with many pointers to the relevant literature:

Heimann, Tobias, and Meinzer, Hans-Peter. Statistical shape models for 3D medical image segmentation: a review, 2009.

Long before statistical shape modelling became a popular tool for image analysis, statistical shape models were studied as a branch of statistics. A good introduction to the statistical aspects of shape modelling can be found in the following classical book:

Dryden, Ian L., and Kanti, V. Mardia. Statistical shape analysis, 1998.

Model-based image analysis and pattern theory

In this course, we have shown you how you can perform model fitting using a simple iterative algorithm. The problem of fitting would deserve much more attention and although these simple algorithms can get you started, more sophisticated methods are needed to successfully apply model fitting to image analysis. We refer again to the paper by Heimann and Meinzer for a overview of different methods used in model-based image segmentation:

Heimann, Tobias, and Meinzer, Hans-Peter. Statistical shape models for 3D medical image segmentation: a review, 2009.

The problem of model-based image analysis can be seen as a special case of the pattern analysis problem. Pattern theory provides a very general framework for the description of patterns based on probabilistic models that are then fitted to the data using an analysis-by-synthesis approach. We recommend the following book by D. Mumford and A. Desolneux, but remark that the mathematical prerequisites are much higher than what has been required in this course:

Mumford, David and Desolneux, Agnès. Pattern theory: the stochastic analysis of real-world signals, 2010.

In our research group we are following this general analysis-by-synthesis approach. Specifically, we have implemented a fully probabilistic fitting framework using Markov Chain Monte Carlo methods. We have recently produced an interactive tutorial on our probabilistic fitting approach. The tutorial is designed to be a continuation of this course. Although the approach is illustrated on the problem of fitting 2D face photographs using a 3D statistical face model, the concepts are general and can be directly applied for analysing medical images.

Scala

If you continue working with Scalismo, it is essential that you learn more about the programming language Scala, as you would eventually like to use Scalismo as a library that can be used in your own applications. There are many good books on Scala available. We particularly like the book:

Welsh, Noel and Gurnell, Dave. Essential scala, 2011.

Another good introduction to Scala:

Cay, Horstmann. Scala for the impatient, 2012.

whose first chapters covering the basics of the language are available here.

Software packages

Shape modelling is very much an applied discipline and software plays an extremely important role. I hope we could convince you that a system like Scalismo, which can directly visualise what is going on, is of great help.

Note that the Scalismo version used for this course (v0.10) is not the latest version. This is due to the fact that version release speed of software is rather high, which would require shooting new tutorial videos frequently. On the Scalismo webpage you will find the tutorial articles that you worked on during this course translated to the latest version of Scalismo. The tutorials will also help you to switch to Scalismo as a library, which we recommend when you are working on real projects.

We have recently also released the software library scalismo-faces, which specifically targets the application of face image analysis. It contains a complete computer graphics pipeline and complements scalismo with functionality for handling color mehes and 2D color images.

If you prefer to use R instead of Scala, you can use the package Morpho together with RvtkStatismo. These packages together offer all the functionality that we covered in this course as well as great visualisation capabilities.

If you liked modelling with Gaussian Processes, but prefer working in C++, you can use Statismo. It supports all the concepts we have taught in the course and is designed to work together with the popular open sources image analysis libraries VTK and ITK.

Data

Maybe the most important part in shape modelling is a set of representative example data from which the shape variations can be learned. The SICAS Medical Image Repository (SMIR) contains a large number of images and surfaces of anatomical structures, many of which are publicly available.

An ideal data set for shape modelling is a set of faces. Besides being great fun, modelling faces also lets us understand and visually assess the plausibility of the modelled shape variations. The Basel Face Model is a face model built from a set of 200 face scans. Besides the statistical model for shape, the Basel Face Model also contains a color and an expressions model. Among many other applications, you can use it for generating realistic face examples for your own shape modelling experiments. It can be downloaded and used freely for academic and non-commercial purposes.

Note: by following the above links you will be taken to third party websites. On these websites you might be asked to submit some information about yourself. Please make sure you are familiar with the terms and conditions and privacy policy of the third party websites before submitting your information. Whether you follow the above links and submit your personal information or not, your course progress will in no way be affected. Nor will your course marks or FutureLearn profile. For the purposes of any personal information submitted the respective third party website will be the data controller, not FutureLearn nor the University of Basel.