Selecting the experts

A critical question on how to implement Delphi Method is how we select the experts? Because the quality and the quantity of the experts is definitely going to influence the reliability of the results. So what are the good tips that we need to follow to select our sample of experts? First of all, experts need to be knowledgeable on the areas that we are researching. It is good to have experts that they are internal, but also external to the firm or to the organization who is actually conducting the research. It’s interesting to find about predictions of the future.

It’s also good to have experts who are coming from various fields, different perspectives, different countries, different religions and different disciplines. This is important because we really need to get a very good representation of different opinions. And of course, to avoid any bias that might come from any different background.

What is also important to note,of course, is to make the experts aware that they’re going to stay anonymous. Nobody’s going to find out who they are, what are their backgrounds. And of course, people are not going to get information about the profiles of any other experts on the panel as well.

How many experts should we have on the Delphi panel? That’s a very good question to answer. Usually theory says that the good sample has between ten to thirty participants. But practice shows that good Delphi studies can also be implemented with a number of experts between fifteen and twenty. Another critical question related to how we implement Delphi studies. The question about how many rounds should we conduct? There is no golden rule or principle to follow here. What theory says is that the number of rounds heavily depends on what the researcher wishes to achieve, what are the stated objectives. Is it to achieve consolidation and consensus? Is it to reach a specific number of rounds?

This is what the research that needs to state from the beginning.

There is an example of an implementation of Delphi study. The purpose of Delphi study was to ask particular experts about what are the topics and the concept that influence experts and professionals in tourism industry to collaborate together. So what the research has done here is to ask different types of tourism experts, managers, directors, even customers. In a destination, researcher wants to undertsand what are the important factors that make tourism professionals and companies to collaborate in the destination. They came up with different topics. And then researchers also ask the experts to give their opinion about the importance of every concept. And this is why you see a score is related to the method.

And they use a scale from one to eight.

Now, in this case, they have implemented two rounds and their criteria is also to have two rounds. It was basically on the rationale that the researchers here, they wanted to achieve consensus about what concepts they had to keep into their study. On the first round, most of the experts

they actually agreed that they have to include the concepts here as the benchmark to achieve consensus. It was to have more than eighty percent of the experts to agree on the inclusion. Now, the concern is related to the degree of geographical location in the first round It was excluded, because even if it had more than eighty percent consensus of the experts, that they can’t be included. The mean value was considerably quite high. Six point nine, which meant that even if people agreed that this concept has to be included, the score of six point nine, that it is a mean score, reflecting high disagreement of inclusion. It increases suspicion of whether to include the concept or not.

So in the second round, when the researchers gave the feedback back to the experts, they reported about this finding that this concept had a consensus of the majority, more than eighty percent of the experts, but with the high mean about disagreement whether to be included. On the second round, the experts, they read this feedback. And then they decided that it’s better to exclude this concept. That’s why at the end, the final decision was taken, that this factor was going to be excluded from the total factors that they were required for the industry to cooperate. And the same happened with the other two concepts. They had the same disparity, meaning that they didn’t have consensus.

But the mean was implying that the experts were quite questionable or worrying whether to include it or not. In this case, the number of rounds was heavily dependent on researches consensus. And that’s when the researcher stop doing the Delphi Method.