Video-based fall detection in assistive technology

The proportion of the global population over the age of 65 is growing fast. Many elderly people experience difficulties with mobility as they age, becoming increasingly unsteady on their feet. Falling is a major worry, and can affect their ability - and willingness - to live independently. In the US, an older adult is hospitalised due to injuries from a fall every 11 seconds, and every 19 minutes, one of them dies. Injuries caused by falling can bring severe consequences. Even when no significant injury occurs, the fear of falling and self-imposed limitations in mobility can contribute to a lack of confidence and acceptance in society, which can lead to depression. The number of fall-related injuries is expected to at least double by 2030.

So, there is a very strong need for an intelligent solution that reduces the number of falls or that minimises their effects when falls do happen. The majority of falls occur indoors in the home environment. The traditional approach to checking on vulnerable people is simply to have a carer visit on a daily basis. However, this can be very expensive, has privacy implications and may cause critical delay if a fall happens long before a scheduled visit. There have been commercially available fall response systems for several decades. Such systems require the fall victim to press a button to reach the emergency services. This may not always be possible, for example if the person is unconscious or unable to reach the button.

An automatic and intelligent fall detection system that is based on visual monitoring avoids the need for user interaction. But to make sure it doesn’t miss a fall and doesn’t report falls when none have happened is a challenging computer vision problem that requires the use of the latest deep learning based approaches. In a home environment, a fall event can be detected with a vision-based system by training a deep convolutional neural network or CNN as a binary classifier to distinguish between a fall and any other activity of daily living like lying down, sitting down or walking. Distinguishing between falls and these normal daily activities is difficult and single images are rarely sufficient.

In contrast to object recognition in single images, actions such as falling are dynamic events that happen over time. The difference between falling and lying down may simply be the speed at which it happens. This means that we cannot simply process individual frames of a video one at a time - we must consider motion. There are two strategies for doing this with a CNN. The first is to process multiple blocks of frames simultaneously - in which case the CNN itself can learn to exploit motion cues by learning filters that find visual features across time. We call this a space-time representation. Alternatively, motion information can be explicitly estimated from the video and given to the CNN as another channel of information.

Providing a CNN with a suitable video input representation and sufficient training data has enabled the successful detection of falls in video sequences.