Skip main navigation

How does Augmented Reality work?

How does Augmented Reality differ from Virtual Reality? Let's explore AR in more depth, and discover the myriad of different uses.
A smartphone with the homepage of Pokemon Go! on screen
© University of York; Anna Bramwell-Dicks

In the last step, we looked at Virtual Reality experiences and got a taster for how VR technology works. At the end, we discussed the Reality-Virtuality Continuum and defined the concepts of Augmented Reality (AR) and Augmented Virtuality. In this article, we’re going to look at AR in more depth.

So, we know that AR integrates digital information into a real-world environment. AR allows users to:

  • Move freely, without fake locomotion
  • Experience expressive interaction with real, physical objects
  • Socialise and collaborate with others in real space
  • Make strong connections between digital content and objects that exist within the physical world

From a technological perspective, it means that developers only need to create assets that augment the physical world, and not the entire environment

To create a convinced AR experience, we need a method to display digital content to the user so it appears to augment physical space, and a method to register that content so it corresponds with features of physical space.

Some examples of AR experiences include:

For these AR experiences to work – where virtual content takes physical space into account – we need an accurate 3D representation of the environment, and knowledge of the user’s position and orientation within that environment. There are numerous different ways to approach this from a technological perspective, including: device pose, Fiducial markers, body tracking and SLAM (Simultaneous Location and Mapping.).

Pokémon Go uses the device pose approach, where the virtual content is kept in the same coarse location by tracking the pose (i.e. orientation and rotation) of the device.
The IKEA AR Catalogue uses Fiducial markers where known images are placed within a camera frame, and by looking at the shape of the marker in the camera image we can calculate its position and orientation.
The FXMirror uses body tracking where the user’s limbs are tracked using a depth camera, and content is them overlaid onto a skeleton.
Finally, the Hololens Project X uses SLAM – Simultaneous Location and Mapping. With SLAM, a depth camera captures a 3D scan of the area of the environment that the user is currently looking at. A 3D model of that environment is then constructed, and the user’s location and orientation within the environment is computed.

All of these approaches work to augment the physical world with virtual content, and each approach has various advantages and disadvantages. But, ultimately, the end result is a mixed reality experience that combines the physical and virtual worlds.

Now that we know how VR and AR work, in the next step we’ll delve deeper into finding out how museums and galleries are using this technology to allow us to engage with, and better understand, the past.

Over to you

Have you ever used Augmented Reality? What did you think of the experience? Do you have any ideas of how these new technologies of AR and VR could be used to create exciting, engaging, immersive storytelling experiences?

As always, share your thoughts in the comments below!

© University of York; Anna Bramwell-Dicks
This article is from the free online

Lights, Camera, Computer - Action! How Digital Technology is Transforming Film, TV, and Gaming

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education