Evaluations and performance review

If most money for public programmes originates from taxes, it’s natural that we would like governments to tell us what we get out of those programmes.

Answers to our questions typically come in the form of evaluations of public policies and programmes. These are assessments of the merits of each programme on the basis of evidence about its results or effects. This evidence often comes from performance reviews, where data about benefits and other results are collected, analysed and reported to public authorities and to the public.

To better understand evaluations and performance review, it’s important to look at the distinction between outputs and outcomes of spending on public policies and programmes.

Outputs are the tangible and immediate results of public services: pass rates for final exams; arrest figures for the police, for example. Outcomes, instead, are the overall effects that originate from the provision of public services. Examples of outcomes include improving levels of education, or providing a safe, crime-free urban environment.

Outputs of public spending programmes are fairly easy to assess. They can be measured by collecting data about the quantity and quality of the services. What really matters, however, are the outcomes. Outcomes relate to the most substantive and long-term effects of public policies and programmes. For example, the outcome of education is to enable individuals to find good jobs, to take their learning to further levels and to be engaged in society in a meaningful way.

Measuring the outcomes of public programmes is harder (conceptually and operationally) than measuring the outputs. For example, how can we measure the outcome of school teaching? Having passed exams and received good marks is not the same as finding good jobs, furthering one’s studies, or engaging with society – all plausible effects of good education that might only be apparent some years after completion of a course of study. The same applies in the case of crime and arrest figures. If we expect our police service to foster a safe urban environment free from crime, then the lack of arrests seems a plausible indicator of this outcome. The number of arrests, however, could be perceived as an indicator of the output (the immediate and tangible results) of policing, rather than an outcome in itself. And what do those outputs actually tell us anyway? Do fewer arrests mean that the urban environment is safe or that the police are just lazy? Perhaps indicators for crime prevention activity, such as the number of patrol hours, are more meaningful than arrest figures alone.

So, how does this relate to us? Often, evaluations and performance reviews are linked with schemes that hold organisations and individuals to account for those very outputs and outcomes. Assessment schemes, where managers and employees are expected to attain certain performance goals in return for additional salary (‘pay for performance’) or better career prospects, are a case in point. Failure to meet the goals may result in penalties for under-performers.

So do these incentive schemes actually result in increased efforts to attain goals (and to improve the performance of public sector organisations)? Poorly designed and administered schemes can have a detrimental effect on employee morale, even triggering counter-productive behaviour: for example, teachers skewing exam classes and results in a bid for performance related salary increases. On both a public and personal level, it’s clear that incentive schemes should be handled with care.