Typology of Innovation Indicators

Innovation indicators are particularly used to allow for comparisons between different countries, that is, on the national level. In order to elaboarate on how a national entity changes or maintains its innovative edge, roughly two sets of innovation indicators can be distinguished: input and output indicators.

Input Indicators:

Input indicators are indicators measuring the financial resources invested for research and innovation. This can be financial resources spent on personnel engaging in research and development, but it can also be financial resources devoted to research infrastructures, material, or software licenses.

In advanced western economies, by far most financial resources are spent by the business sector, with particular sectors being particularly prone to invest in R&D activities, such as the pharmaceutical industry. The state and higher education institutions invest significantly less in research and innovation (but steadily increased their budget in nominal terms). In Germany, for instance, about two thirds of the total investments are made by businesses, while the state and higher education accounts for another third.

The generation of input data is again more complicated than it sounds, since the state has no rule for forcing the companies to report on their activites. Therefore, data about R&D investments are generated by large surveys relying on self-reporting of the companies. In Germany, for instance, there is R&D survey in the business sector (FuE Erhebung in der Wirtschaft) conducted and reported by the Stifterverband für die Deutsche Wissenschaft which is currently considered the most reliable information about innovation investments in the business sector in Germany.

Two indicators are particularly often used for policy making and reporting. The first indicator is called the GERD indicator (Gross domestic expenditures on research and development) and relates the expenditures both by the state and business entities. Similarly, the Higher Education Expenditures on Research and Development (HERD) measures the expenditures in the university sector as a share of GDP.

These input indicators for research and innovation are attractive for policy making, because they allow for defining a clear policy goal that can be reached by increasing spending dedicated to R & D. More than twenty years ago, the European Union, for instance, defined the goal of investing 3 % of the EUs gross domestic product (GDP) into research and development. In Germany, the 3% goal for GERD has been reached since 2018.

Shortcomings of Input indicators:

Input indicators for research and innovation can be criticized because they also largely depend on the labour costs for these activities in the countries. For instance, doctoral researchers can be paid higher or lower salaries as a function of their demand or as function of the political willingness to acknowledge their contributions. In the past, countries such as Sweden performed well in the GERD indicator ranking partly due to the comparatively higher wages for younger scholars.

How input indicators are used has been also criticized. The above described indicators measure only how many resources are spent for R&D activities. In public debates, however, investments in science and innovation have often been conceived as an expression of their competetiveness, which tells us very little about the actual performance of research and innovation. The UK, for instance, spent just under 3% of its GDP for R&D, but is considered a leader in science and innovation.

Output indicators:

Output indicators, however, are performance indicators aiming at measuring the actual performance of research institutions and businesses in research and innovation. Most internationally relevant output indicators operate on the national level in order to advise governments, as well as trade and business unions in matters of innovation policy and to allow for international comparisons. Output indicators are mainly available for a rather restricted set of outputs:

• Publications,
• Patents, and
• New products

Publications:

In order to assess the research performance of a given country, the number of publications is counted. In order to perform comparative analyses, the number of publications is related to the population in a given country. The relative publication output, for instance, counts the output of publications per 1.000 inhabitants. Because of its research intensity, smaller countries like Switzerland or the Netherlands are leading in this regard (Grupp & Mogee, 2004).

Data for measuring research performance are provided by large bibliographical databases such as Web of Science (run by Clarivate) or Scopus (owned by Elsevier). Both are research information services that also provide metadata about the publications, for instance, the data of publication, the source, the authors and affiliations, as well as, most importantly, the number of citations that a given publication receives.

Nevertheless, the large bibliographical databases have also been critisized for their bias towards English language and internationally oriented research output (Mongeon & Paul-Hus, 2015; Moya-Anegón et al., 2007). Bibliographical databases cover mostly the research output in peer reviewed publication outlets, such as journals or in some cases proceedings of conferences. Books for instance as well as research that is not published in English are less covered. The problems of coverage have been discussed repeatedly among the bibliometric community and data providers constantly have made efforts to improve.

Also, it is not always easy to decide which country a given publication is counted as given the difficulties of disambiguating affiliations. Despite these shortcomings, however, the relative measurement of publications remains a central indicator for understanding the research capabilites of a given country.

Patents:

Patents are also a relevant source for understanding innovation capacities. This is because patents are a defensive title that are only granted to an inventor by an authority (be it the European Patent Office, the US Patent Office or the Japanese, just to the name the most important) if the novelty can be sufficiently proved against the state of the art in technology and research. Hence they can be perceived as an indicator for measuring the novelty of technology. Yet, according to Joseph Schumpeter, these should not be termed innovations, as innovation is associated only with the successful introduction of a novelty in the market or society. Again, the total number of patents are counted by referring to the country of origin of the inventor. Patents on the national level are also counted in relative terms, that is, in relation to the population of the country (Grupp & Formahl, 2010).

More recently, however, the information of patents as an indicator for inventiveness became more contested, as the number of patents in some particularly fast growing technology and business fields such as the software industry decreases. Moreover, the process of patenting is very costly and therefore can more precisely be interpreted as an expression of the company`s willingness and capacity to invest in securing its intellectual property.

Because of the costliness and duration of this process, for instance, in Germany most patents are granted to the large companies, such as Bosch, Daimler or BMW (Rainer Frietsch, Peter Neuhäusler, & Oliver Rothengatter, 2013). Moreover, there is a strong bias towards the automotive and – to a lesser extent towards the chemical industry. Patent data are usually provided by the Patent Offices themselves, but are processed and made available in large electronic resources such as PATSTAT. Because of the difficulties of the granting procedure, it takes expert knowledge of the process to interpret these data.

Products:

Finally, the most substantial output indicator for innovation is the introduction of new products and services. Yet, this is also the most difficult to measure, since – as mentioned above – it is not easy to decide what can be considered novel. The source for this information in the realm of research and innovation are the companies themselves who report in large surveys about their new products in different realms of their operation.

In Europe, there is a large survey known as “Community Innovation Survey” (CIS) that generates company information about innovation in firms first implemented in 1992. Firms were asked about their product or process innovations, about barriers to innovate as well as about collaboration with industry and science. In Germany, the most advanced survey is the Mannheim Innovation Panel or reports within the Mannheimer Unternehmens Panel (MUP), both of them are considered representative in terms of business participation. In innovation research, the surveys are at times also criticized because of the problem of self-reporting. Yet, still these data are the most substantial regarding innovation output and therefore considered relevant.

Shortcomings of Output Indicators:

More recently, however, output indicators have been critisized, because they usually refer to only a select set of research and innovation activities, and appear increasingly not to reflect changes in innovation and invention behaviour.

First, the trend towards digitalization has largely accelerated development processes leading to shorter innovation cycles. As mentioned above, patenting appears to be less attractive in some domains, as it is extremely time and resource intensive. Moreover, there are other types of invention activities that are not covered by the rather distant measure of patents, such as software releases in the IT industry.

Second, the different output types still imply that somehow publicly funded research is the most relevant input to business innovations. This is also the case for so called science intensive patents, as these are measured by the extent to which the justifications for the novelty relies on citing scholarly publications. There are certainly sectors in which the science dependency is still increasing, yet there also other sectors such as knowledge intensive services, which are largely transformed by the application of artificial intelligence which are not covered.