Skip main navigation

Phase 2: piloting and evaluating

In the following video, Prof. Roelien Bastiaanse depicts how the language test is piloted and evaluated after designing it.
Now that you have learned how we create an item list, we will show you how we evaluate whether it stands our requirements and what we still may have to revise. The idea behind pre-testing or piloting a test is that you take a test run first. Instead of going straight to your patient group, you pilot your test first in a healthy population. This is crucial to avoid wasting patients’ time with an unfinished test. We first want to know whether healthy speakers give the desired answers. In a small group, you will run your tests under the same conditions as you would run your test with the patients. In this way, you can evaluate what happens. Is the test too long?
Maybe it’s even too exhausting for a healthy person, let alone for the patient group. Does concentration last that long? Is the set up correct? Do you need to move to screen? The recording equipment? Is it comfortable for the participant? And even if you plan very carefully, you need to expect unexpected events. Piloting is therefore crucial to run into those problems before the actual testing. And most importantly, the piloting helps you to answer the question, do the items work as intended? Once the results are in, you can revise your list of items. Do some of them need to be omitted because they could not be named fast enough, accurately enough? Do some pictures need to be redrawn?
As a rule of thumb, consider to only keep items in the final version if the picture was eliciting the expected answer in 90% of the cases. In a clinical setup, the test will then go to the next step, standardising your paradigm in an even larger, healthy population of participants of both genders with a similar age and education, with the, in our case, brain tumour patients.
In the following video, Prof. Roelien Bastiaanse depicts how the language test is piloted and evaluated after having designed it.
This article is from the free online

Language Testing During Awake Brain Surgery

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education