Interaction in AFrame

With a simple introduction to interactions in VR, let’s see how these are applied to an example in AFrame.

Interaction in AFrame
AFrame supports a wide range of input controllers and hardware. The creator of a VR experience can use the underlying API to instantiate the controllers and receive specialized trigger events. The virtual world can be programmed to respond to these events. In this session, we will explore how AFrame handles touch/click and gaze-driven interactions.

Page-level Interaction
We have already seen page-level touch/click interaction. If you recall the Newton’s Cradle example from last week, the virtual world listened to click/tap anywhere on the page. It responded by updating the rotation of the table (either start or stop). The virtual world has no idea which entity was actually clicked on and always responds in exactly the same way. Clicking on the leftmost or the rightmost ball produces the same effect as clicking anywhere else on the page.

Ray-cast Cursor Interaction
The raycast cursor component of AFrame makes the click behaviour more realistic and immersive. Let’s revisit the Newton’s Cradle example and update it to include the raycast cursor. We have provided the updated file below (See “Interactive AFrame – Newton’s Cradle” below). If you look at the source-code, you will see that the raycast cursor is defined as a subset of the camera entity as follows:

<a-camera look-controls
 mouse-cursor cursor="rayOrigin: mouse;"
 raycaster="objects: .clickableBall, .table">
</a-camera>

The code snippet above defines a mouse/tap based ray-cast cursor.
Every user click or tap generates an event.
For each of these events, AFrame draws a ray from click location into the scene.
It searches for interactive (i.e. clickable) objects that lie in the path of the ray.
If an interactive object is found, AFrame dispatches a click event to it.
The click event handler of the object takes over and responds to the user click.

Ray-object intersection is a computationally expensive process.
Thus, the designer should filter out all non-interactive objects.
AFrame supports HTML query-selector style object specification and filtering using the objects property.
To select a group of objects with same class name, use the ‘.’ prefix.
To select a specific object with a known id attribute, use the ‘#’ prefix.
In our example, we only check objects with class name ‘clickableBall’ and ‘table’.

We define their interactive behaviour through the functions ‘setTableSpin’ and
‘setFirstSwing’.
The table doesn’t start spinning till it is clicked.
Similarly, the outermost balls don’t start swinging until the user clicks on one.
The ball which is clicked swings first.
From an interaction perspective, this example feels more realistic than the original example.

Gaze Cursor Interaction
Gaze-cursor is an interesting interaction method which relies on head-pose. The gaze cursor appears as a circular reticule in the center of the screen. On a smartphone, the gyroscope controls the cursor movement. The scene moves opposite to the movement of the device, while the cursor stays fixed to the center of the screen.

The cursor can be configured to ‘fuse’ with a target using ray-casting. An imaginary ray is cast into the virtual world from the reticule. If the ray hits an interactive target, the cursor goes into fuse mode, shown by a shrinking reticule. If the target is still in contact when the fuse-delay ends, a click interaction is registered. The fuse-delay prevents unintentional clicks on interactive objects.

We have provided an example (See “Interactive Gaze – Newton’s Cradle” below). Explore the behaviour using a smartphone. The key piece of code that sets up the behaviour is:

<a-camera look-controls mouse-cursor>
 <a-entity id="cursor" raycaster="objects: .clickableBall, .table"
 cursor="fuse: true;" 
 material="color: black; shader: flat"
 position="0 0 -2"
 geometry="primitive: ring;radiusInner: 0.16; radiusOuter: 0.24;" 
 animation__click="property: scale; startEvents: click; easing: easeInCubic; dur: 150; from: 0.1 0.1 0.1; to: 1 1 1"
 animation__fusing="property: scale; startEvents: fusing; easing: easeInCubic; dur: 1500; from: 1 1 1; to: 0.1 0.1 0.1"
 animation__mouseleave="property: scale; startEvents: mouseleave; easing: easeInCubic; dur: 500; to: 1 1 1"
 >
 </a-entity>
</a-camera>

The code is longer but easy to understand when processed in blocks. The <a-camera> is similar to the previous example. However, the cursor control is now a separate child entity. The raycaster attribute specifies the objects to look for ‘clickableBall’ and ‘table’ class objects.

The ‘cursor’ attribute with a property called ‘fuse’ set to true enables fusing behaviour. The next three attributes define the cursor’s shape and form. Finally, the animation attributes define the cursor animation in response to interaction events. AFrame sets the default fuse duration to 1500 ms. To change it, add the ‘fuseTimeout’ attribute with a different value. Test different time-durations to achieve a good balance of usability versus false triggers. Gaze-cursor can also work on laptops and desktop devices. The gyroscope motion is replaced by mouse dragging action.

Interaction is an important tool for users to engage with a virtual world. Never be tempted to design for a specific input device only. The diversity of input devices means a user may not have that specific input device. A good interaction workflow allows user to interact with whatever they have available. The user experience will improve if the virtual world provides a wide range of support for performing the same action.