Multi-level/multiple sources of data collection

And the rest of two topics, I just want to touch the base, because, for example, this multilevel/multiple source of data collection is very high level study. That’s a long story. Now we use the experimental design, use the scenario-based studies, SBSs, use the virtual reality and so on. But people still think that your only source of data could not have the real or true value. This happened because of the Journal of marketing, which is topic journal in the marketing field. I think it was about five, ten years ago, the editor-in-chief said I would not accept any single study, which means you can’t only use one source of data.

(The editor-in-chief said) I will not accept any self-report research, you have to have experimental components in your study. The push part actually is the article published in quite many years ago, 20 or 30 years ago, by two psychologists. And they did report about CMV, which is common massive variance, which is the cause of self-report study, the cause of the single level or single source data collection. So that comes up with this topic, how can we use the multi-level? What do we mean the multi-level? For example, if you do a study, particularly the restaurant study, you have customer, people serve you, and supervisor, you have three different levels. Usually we do study, we only ask customer how satisfied they are.

But we did not ask employees, so there’s only a single level. But if we involve employee into the study, we have two levels. If we involve the supervise together, we have three different levels. Three levels, three sources, in each level we collect the data independently, so we have three levels, three sources of data collection, and try to avoid CMV. One of my doctor students and she did a study, I would like to share with you. It’s a very good study and she already published three papers on top journals. She used the multi-level, multi-source data, but it took a lot of time.

That’s CMV, I think I mentioned that to you, the reason is why, you only use a common rater, which means the questionnaire measurement, the manner in which items are presented to the respondents, and the context in which items are on a questionnaire are placed in the context inference. Your respondents are probably influenced by the time, which means you may recall three months ago, you recall six months ago, you recall yesterday, or you recall immediately after your visit, and by location where you did it and by media. Those are all the things that actually cause CMV. These are the reference published by a few psychologists, they identify CMV.

They are very good articles, if you want to know more about CMV, you may check these journal articles. There are four remedies of CMV. One is using other source information, other source means other data source, not only one source, but other data source. The second, mixing the order of questionnaire to use a different scale type to eliminate the possible cause of CMV. Sometimes we use the questionnaire ordered by … and the measurement scale is all one to five Likert scale. But if you mix the order or maybe use different scale type, probably you can eliminate, not totally eliminate. Probably you can reduce the CMV by using other source of data.

No.3, complicated specification of regression/SEM model, such as if we add mediating, moderating, or maybe non-linear guided by good theory, probably you can also eliminate or reduce the CMV caused by the self-report study. No.4, the statistical methods to detect and control for any possible CMV. I talked with a few journal editors and I said to them, for each paper if it is self-report study, we mostly require the author to test or to detect if CMV seriously concerned in the study. There are 4 different ways you can do it. One is Harman’s single-factor test. It’s very simple and it can tell you the possibility the CMV will be a serious concern in your study. Next one is the partial correlation procedure.

Next one, the direct measure of a latent common method factor. The last one, use multiple remedies to find out. Just like the previous one we said, you can use multiple-source data, you can use different order or different measurement scale to try to eliminate or reduce the CMV. The above I just described a little bit of the very beginning, as a multi-level/multi source data collection, the purpose is to minimize the CMV, the common massive variance. Data are collected from different sources, supervisor, employee, customer. And you also have a level of team-level or group-level, you have different levels approaching probable topic, m Maybe it will provide general support for the authors’ hypotheses.

That is a very good way to eliminate or reduce the impact of CMV of self-report study. This study was done by one of our doctor students, Gloria Liu. She did a study in the restaurant setting. She did a very big study, she has several hundred customers and a couple hundred employees together as the two groups. One is the individual group, one is unique group. She combines the data. For example, one restaurant employee serves three customers. The three customers fill in the questionnaire, and the employee also fill the questionnaire for these three customers. So that’s a cluster, one class for employee, one class for the customers. At the very beginning, she wanna add one more level.

I told her please don’t do it, it’s very complicated. Then you look at the model, it is actually structure equation modeling. But this modeling is not only SEM, because you need to build group level and individual level together. This is what we called the multi-level SEM, which is right now I should say the highest, most difficult statistic study and the statistic procedure used, because the multi-level data is used to build SEM to run together. That study actually is very good study. They collected customer data, employee data, even the manager data together.

That is what we called the multi-data, multi-source data, multi-level data, because they had different levels of study. Remember the purpose of that study is to reduce or eliminate CMV, which may cause the bias for the study. Next one, I just very simply go through about the longitudinal study. Basically, when we do study, we have two different sections. One is called the longitudinal study. One is called the cross sectional study. A cross sectional study, which means we cut the sample and do the one study at one time. But a longitudinal study is a fixed sample of elements that measured repeatedly through time.

The time could be monthly, could be weekly, could be quarterly, could be biannually, could be annually, or maybe two years, three years or whatever. There’s repeat again within certain period time you set up. Longitudinal studies have two set categories. The first one, we call the true panel, which means in each measurement and study, we always keep the variable the same, we don’t change variables. The next one, we call the omnibus panel, which means we change the variables from time to time. This time we change a couple of measurements, next time we change a couple of measurements, that’s longitudinal. Longitudinal study is a good study if you want to see the trends.

How do people change their consumption behavior, how do people change their attitudes, how do people change their perceptions? Really you can see the trends. But I do not suggest students to do it, because that will take a very long time. You have to measure at least two or three times, that really delay your study. But when you graduate, I think it’s a good way, particularly if you want to see the developing trends. I think that’s a very good study actually, but it’s costly, because you need to always keep the panel of people size. If you say the sample size is 500, you will always keep your 500 people size every time.

So it is really a challenge that keeps all these repeatedly from time to time. But longitudinal study is a good study for catching the trends of people’s behavior or consumption.