Skip main navigation

New offer! Get 30% off one whole year of Unlimited learning. Subscribe for just £249.99 £174.99. New subscribers only T&Cs apply

Find out more

Phase 2: piloting and evaluating

In the following video, Prof. Roelien Bastiaanse depicts how the language test is piloted and evaluated after designing it.
8.2
Now that you have learned how we create an item list, we will show you how we evaluate whether it stands our requirements and what we still may have to revise. The idea behind pre-testing or piloting a test is that you take a test run first. Instead of going straight to your patient group, you pilot your test first in a healthy population. This is crucial to avoid wasting patients’ time with an unfinished test. We first want to know whether healthy speakers give the desired answers. In a small group, you will run your tests under the same conditions as you would run your test with the patients. In this way, you can evaluate what happens. Is the test too long?
51.7
Maybe it’s even too exhausting for a healthy person, let alone for the patient group. Does concentration last that long? Is the set up correct? Do you need to move to screen? The recording equipment? Is it comfortable for the participant? And even if you plan very carefully, you need to expect unexpected events. Piloting is therefore crucial to run into those problems before the actual testing. And most importantly, the piloting helps you to answer the question, do the items work as intended? Once the results are in, you can revise your list of items. Do some of them need to be omitted because they could not be named fast enough, accurately enough? Do some pictures need to be redrawn?
98.8
As a rule of thumb, consider to only keep items in the final version if the picture was eliciting the expected answer in 90% of the cases. In a clinical setup, the test will then go to the next step, standardising your paradigm in an even larger, healthy population of participants of both genders with a similar age and education, with the, in our case, brain tumour patients.

In the following video, Prof. Roelien Bastiaanse depicts how the language test is piloted and evaluated after having designed it.

This article is from the free online

Language Testing During Awake Brain Surgery

Created by
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now