Filmmaking: The History of Tracking Technology

Tracking can enable filmmakers to attach objects to camera moves, and this can be really powerful in the creation of amazing composited shots.

In fact using tracking and its by-product stabilising is a daily occurrence for most VFX artists, so it’s worth getting familiar with the technique. Of course we’re only scratching the surface of this technique here, there are more complex processes like 3D and Planar tracking, but for now, the HitFilm exercise is a great start.

Ensuring the flare moves convincingly in sync with the actor moving the stick would be hard to achieve without tracking- you would need to approximate frame by frame and that would end up looking very wobbly.

In fact, historically this effect would have been done by refilming shots or what were called ‘plates’ under a film rostrum camera frame by frame, and this was never wholly convincing, because our eyes are hyper-critical of every wobble or misalignment on film. Back in those celluloid days tracking was done by hand, and someone would need to move an image of the flare frame by frame onto the stick under a rostrum camera. It was only when computers allowed us to track pixels that smooth and convincing attachments were really possible.

In today’s digital world we let the computer do the donkey work- the tracking software looks for the same shape or a similar group of pixels in every ensuing frame and then attaches the flare.

A little bit of history

This ability to digitally attach one object to another is relatively new. Like a lot of technologies we use everyday, you probably won’t be surprised it has a military origin- in missile guidance systems. However it wasn’t long before enterprising post production companies in the early 90s saw the research papers and adopted the technology.

ILM (Industrial Light and Magic, the company originally behind Star Wars) first tried a form of laborious manual tracking on 1992’s Death Becomes Her (dir: Robert Zemeckis). The film used extensive digital retouching to remove the head of one actress, and then later placed a talking head on another body. The head needed to be tracked in place. ILM would improve on this to create a ‘3D’ tracking system that let them calculate depth so they could track and place computer generated dinosaurs in three dimensions, (not just two dimensions like we’ve used in the flare shot where we only move up/down and sideways)

It was clear in the 1990s that the public’s appetite was moving towards more spectacular and fantastic themes, so there was a demand for hardware that would allow such magic and a company called Discreet built a system called Flame in 1992 that had an inbuilt tracker. The first film that Flame created VFX for was the Super Mario Bros, (dir: Annabel Jankel and Rocky Morton).

In the 1993 Sylvester Stallone movie Cliffhanger (dir: Renny Harlin) tracking was used to cover wooden supports on the studio set of a cliff-face, by aligning rock images over the visible wooden struts and carpentry.

From our privileged position these days it’s hard to appreciate what a breakthrough this was. It was magic! As Peter Webb, an early Flame Artist said of those times;

“I remember demoing the tracker myself in the early 90s at trade shows and we always used this footage of some Russian soldiers marching. We would track the Russian emblem on their fur hats and then replace it with a flame fish (the old Flame logo). The punters LOVED it as it had never been seen before”.

By 1994 other systems were adding tracking to their range of functions. Various processing techniques were invented- changing the luminance on the pixels so they were easier to follow, or tracking Red, Green and Blue channel values separately and then combining them to average out errors. One common problem with tracking a group of pixels over time is that they change. Also if your film footage is grainy, the computer isn’t intelligent enough to recognise this so ‘loses’ the group of pixels it was tracking. By 1995, machines allowed for the track point (the group of pixels) to be refreshed periodically as it tracked, and the track re-started over so many frames, allowing for longer smoother tracks and therefore more complex and longer effects sequences.

By 1998 the Oscars recognised Flame with a Scientific and Technical Academy award. A “tracking arms race” ensued- everyone wanted to use the best tracker technology for their films. If you imagine a Hollywood film might be four thousand pixels across so it looks good on the big screen- having a tracked object go wrong by just a couple of pixels might be noticeable to the audience, so you can appreciate the commercial importance.

Also, as film goers demanded more sophisticated VFX, more 3D camera and object match-moving systems proliferated. Clever software could read film footage and work out the 3D space it originally portrayed, and then provide accurate data to allow 3D computer generated models to seemingly be placed into moving image shots and perform complex moves as if they were really there. Software like 3D Equalizer and later Boujou revolutionised how VFX could be integrated into filmed footage. Films like The Matrix, Lord of the Rings, Harry Potter and other spectaculars relied on this technology.

Think a minute about this story. What started out as technology for missile guidance systems is now as essential part of the entertainment industry! Without tracking, the VFX in today’s movies would be very limited indeed.