Active contour models for shape tracking

When we are trying to track an object, one way we can do this is by following the shape of the object between frames. To do this, we need a way of representing the shape of an object in an image. For this purpose we can use a family of models called active shape or active contour models.

Active contour models

Active contour models have two main components which allow them to pick out an object. Features such as edges and corners help us to delineate objects in the scene; they allow us to pick out a boundary around an object. If we are building a model to represent shape, using features such as these are helpful to incorporate as an external energy, or external constraint. These parts of the model examine the image data to help adjust the curve to the shape of the object.

The second main component of the model is a representation of the boundary curve itself. The model must be able to describe such a curve, as well as adjust it in response to changes in the object shape, or to allow it to adjust to fit an object initially. Such properties are represented as the internal energy, or internal constraints. They control how flexible the curve is, and how it is able to move.

Depending on the application, the nature of these energies may be different, but, typically, the external energy (otherwise known as an image energy) is drawn to edge features in an image, and the internal energy favours a smooth, well-represented curve. By well represented, we typically mean any key or control points which help describe the shape are regularly space out, rather than bunching up, which can cause knotting-type artefacts.

Left: Initial curve.

Right: the curve fitted to the image data, balancing the pull of edges in the image against how much the curve is allowed to bend.

Control points, which are used in the mathematical model of the curve, are marked as red circles. One of the internal constraints keeps these points evenly spread around the leaf.

Initialising active contour models

Tracking with active contour models

Left (timepoint 1): The contour from the previous frame is overlaid on the current frame image. The active contour model does not yet line up with the shape.

Right: After allowing the active contour model to run and settle, the shape model is now a good fit to the object boundary.

Further reading:

• The original article on active contour models Kass, M., Witkin, A. & Terzopoulos, D. Snakes: Active contour models. Int J Comput Vision 1, 321–331 (1988). https://doi.org/10.1007/BF00133570

You can find lots of example implementations of these models in many programming languages.