Some criticisms of TIMSS

We now want to consider some of the criticisms of TIMSS.

It is not easy to compare countries! Although the TIMSS study is very solid, there are certainly limitations.

These include:

• Some people mention that countries are so different (for example, their culture, population and education policy) that it is almost impossible to compare countries. They argue that the ‘country contexts’ are so unique that international comparisons are meaningless.

• Although it might seem that all participating students sit the same tests, they actually all make a slightly different set of assessment items. If this wasn’t the case, then students would all have to sit an exam over many hours to cover all the content domains. Some critics are skeptical about the extent in which advanced statistical techniques, can take such ‘missing data’ into account.

• When analysing the data, most analysts assume that the item difficulty (i.e. how hard the question is) is the same in all countries. Some critics argue this is not necessarily the case.

• Finally, some people do not mind tests like TIMSS, but mind what decision makers do with the data. They would argue that ranking countries creates unhelpful competition.