Skip main navigation

How does Augmented Reality work?

How does Augmented Reality differ from Virtual Reality? Let's explore AR in more depth, and discover the myriad of different uses.
A smartphone with the homepage of Pokemon Go! on screen
© Photo by Mika Baumeister on Unsplash

In the last step, we looked at Virtual Reality experiences and got a taster for how VR technology works. At the end, we discussed the Reality-Virtuality Continuum and defined the concepts of Augmented Reality (AR) and Augmented Virtuality. In this article, we’re going to look at AR in more depth.

So, we know that AR integrates digital information into a real-world environment. AR allows users to:

  • Move freely, without fake locomotion
  • Experience expressive interaction with real, physical objects
  • Socialise and collaborate with others in real space
  • Make strong connections between digital content and objects that exist within the physical world

From a technological perspective, it means that developers only need to create assets that augment the physical world, and not the entire environment

To create a convinced AR experience, we need a method to display digital content to the user so it appears to augment physical space, and a method to register that content so it corresponds with features of physical space.

Some examples of AR experiences include:

For these AR experiences to work – where virtual content takes physical space into account – we need an accurate 3D representation of the environment, and knowledge of the user’s position and orientation within that environment. There are numerous different ways to approach this from a technological perspective, including: device pose, Fiducial markers, body tracking and SLAM (Simultaneous Location and Mapping.).

Pokémon Go uses the device pose approach, where the virtual content is kept in the same coarse location by tracking the pose (i.e. orientation and rotation) of the device.
The IKEA AR Catalogue uses Fiducial markers where known images are placed within a camera frame, and by looking at the shape of the marker in the camera image we can calculate its position and orientation.
The FXMirror uses body tracking where the user’s limbs are tracked using a depth camera, and content is them overlaid onto a skeleton.
Finally, the Hololens Project X uses SLAM – Simultaneous Location and Mapping. With SLAM, a depth camera captures a 3D scan of the area of the environment that the user is currently looking at. A 3D model of that environment is then constructed, and the user’s location and orientation within the environment is computed.

All of these approaches work to augment the physical world with virtual content, and each approach has various advantages and disadvantages. But, ultimately, the end result is a mixed reality experience that combines the physical and virtual worlds.

Now that we know how VR and AR work, in the next step we’ll delve deeper into finding out how museums and galleries are using this technology to allow us to engage with, and better understand, the past.

Over to you

Have you ever used Augmented Reality? What did you think of the experience? Do you have any ideas of how these new technologies of AR and VR could be used to create exciting, engaging, immersive storytelling experiences?

As always, share your thoughts in the comments below!

© University of York; Anna Bramwell-Dicks
This article is from the free online

Lights, Camera, Computer - Action! How Digital Technology is Transforming Film, TV, and Gaming

Created by
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now