Building Evidence for Technology and Autism (BETA)

Intervention-based research for autism can confuse integration with inclusion. Integration has a focus of supporting the autistic individual to adapt to their environment (eg social skills training) whilst inclusion attempts to adjust the environment to be more autism-friendly (eg reasonable adjustments within the school).

Earlier we saw that within an educational context, whilst most autistic children are integrated within mainstream schools, inclusion can remain an illusion. The same can be true for digital interventions that support autistic people – they can focus on attempting to address perceived deficits in the person rather than the environment – digital intervention instead of digital inclusion (Brosnan et al, 2018). So how do you choose the right digital technology?

Autistic people, their families, related professionals and teachers were asked what constitutes evidence for a project called Building Evidence for Technology & Autism (BETA). There was a consensus on the following three points:

1. The three areas that evidence is required for are:

a) Reliability, which is the efficacy of a product at the level of engineering. Is it technically sound/functional? How well does it work? For example, does their face recognition functionality actually work? Does the app crash often?

b) Engagement, which is the user perception of the technology. How usable, agreeable, pleasant and accessible a product is for the specific users? Its ease of use/look & feel.

c) Effectiveness, which is the outcome of using the product. How much impact does it have to the people using it? Does it make an observable difference in the user’s life/behaviour?

2. There are four useful sources of evidence:

a) Trial version: A trial version is available. Its functions can be explored. Whether it is motivating to use in the short/medium/long term can be evaluated. It is clear how the trial version differs from the full version.

b) Expert/Professional opinions: Opinions about the product have been made by experts (for example, autistic people or other people with specialist skills and relevant experience with technology), relevant professionals (for example a specialist teacher, speech and language therapist, specialist psychologist, etc.), or relevant organisations/agencies (for example an autism society).

c) Online reviews: Review websites and social media groups contain information about the product. Reviews from autistic users and their families have been included. People having used the product for a (relatively) long time have expressed their views. Many reviews are available for comparison to improve objectivity.

d) Academic/scientific research: Academic articles evaluating the product are available. The quality of the research can be established (eg if it has been peer-reviewed). Articles have been reviewed in a way that systematically combines the results from multiple independent studies. Online information or talks in the mainstream media by qualified scientists are available. The academic’s or scientists’ s relevant qualifications, affiliations and potential conflicts of interest can be reviewed.

3. There are four different levels of evidence:

a) Complete, which is high quality available evidence on all (or mostly all) of what is needed.

b) Adequate, which is high quality evidence on some of what is needed, or adequate quality evidence on all (or mostly all) of what is needed.

c) Limited, which is adequate quality evidence on some of what is needed, or limited evidence for all (or mostly all) of what is needed..

d) None, which is no, or not enough, evidence. This includes ‘don’t know’, if you are unaware of any evidence.

Importantly this BETA analysis does NOT identify if the product is good. The BETA analysis identifies the level of evidence available for the product. But this is an essential step in deciding which technology to use.