The reality-virtuality continuum

There are a fair few terms in the immersive media world, and admittedly it can get confusing at times, especially when technology moves fast. There are three main terms used in this field that you should have an understanding of…

These terms are augmented reality (AR), virtual reality (VR) and mixed reality (MR). We’ve already explored AR and VR in the previous steps, but what about MR?

Mixed Reality is AR and VR’s lesser-known cousin; but is probably just as important. Why? Because mixed reality describes the entire continuum from AR all the way to VR. That space in between AR and VR is a pretty exciting and creative place. This step will introduce you to mixed reality and the reality–virtuality continuum.

The reality-virtuality continuum (Click to expand)

This concept was first defined in 1994 at the tail end of VR’s first wave by Paul Milgram; an engineering professor at the University of Toronto. Milgram’s now seminal paper, A Taxonomy of Mixed Reality Visual Displays, expanded understanding of immersive tech as something brimming with opportunities well beyond the binary notions of reality versus virtual reality. In short, Milgram defined MR as visual ‘technologies that involve the merging of real and virtual worlds’.

So what does this actually mean in practice and how can people actually experience MR?

Nowadays most VR headsets are also capable of running mixed reality experiences and this is how the majority of people experience it. There is also a range of dedicated mixed reality headsets commercially available, which run mostly on the Windows’ Mixed Reality platform.

Examples of MR experiences in action:

The VOID

The Void are a Utah-based startup who partner with large entertainment franchises (eg. Lucasfilm and Disney) and create location-based mixed reality experiences. These ticketed experiences allow guests to step into the storyworld of their favourite films ranging from Star Wars to Ghost Busters to Wreck it Ralph. Groups of guests enter the environment together, often united on a certain themed mission. Freely moving around in the space, they embody characters and can touch and interact with their environment. A unique combination of haptic feedback, motion tracking and VR headset technology drives the experience combined with a touch of classic magical illusion.

Mixed reality for an enhanced home

In the future, it is possible we will use mixed reality to map extra functionality onto our existing daily environments. Interaction designer and magician Greg Madison created just this as an example of what the home of the future might look like. He can now use his dining table as a massive display to explore maps on. He can also play the piano with virtual, digital sheet music. His impressive experiments went viral, gathering global attention and inspiring the immersive technology community.

Philips and Microsoft’s Hololens app for surgeons

The partnership pilot between Philips and Microsoft allows surgeons to use mixed reality glasses to see a ‘hologram’ of the organ they are working on, floating in front of them during minimally invasive surgery (aka. keyhole surgery). Usually surgeons have to rely on 2D data on a separate screen, for instance x-rays or ultrasound. The live 3D model in the surgeons’ field of view has been found to be far more intuitive and effective.

It is clear that as an immersive media maker, seeing AR, VR and MR as a continuum opens up a whole wealth of creative possibility.

© Catherine Allen

Have your say

There is much to be discovered across the immersive spectrum.

• Having watched the videos, what are your thoughts?
• Have you experienced mixed reality? What was it like for you?

Share your thoughts in the comments section and hear what your fellow learners may have experienced.