Ethics in Measurement

What you can’t measure, you can’t control! This quote is a frequent dogma of management literature and it is especially relevant in hierarchical structures, where you need to have control over things happening two or more steps down the ladder. But measurement has its downsides.

We tend to believe that numbers are objective, although they can be as tainted and manipulated as a written report. The problem with measurement – especially in connection to public goods – has been widely researched.

Smith (1995), for example, brought to light eight unintended consequences of impact measurement:

(1) tunnel vision (emphasis on quantifiable measures at the expense of other aspects);
(2) sub-optimisation (pursuit of narrow local objectives at the expense of organizational objectives);
(3) myopia (pursuit of short-term targets at the expense of long-term objectives);
(4) measure fixation (an emphasis on measures rather than the underlying objectives);
(5) misrepresentation (deliberate manipulation of data);
(6) systematic misinterpretation of data;
(7) gaming (manipulation of behavior to get rewards for achieving targets); and
(8) ossification (rigid measures to inhibit innovation).

When planning and conducting an impact measurement tool, you should always be aware of the difficulties connected to it. Here are some recommendations to pay attention to in order to get suitable results with high acceptance:

Work in the Interest of Beneficiaries

• Participants must consent to share information voluntarily
  Be sure to obtain consent from participants. Using a standard consent form helps to integrate this step in the general procedures and to be sure that legal requirements are met.
• Be careful with incentives
  While it is appropriate to provide incentives for participating in measurement activities, you have to be aware that this might influence the sample composition and the answers of the participants.
• Sensitive information should be protected through anonymity
  Thoughtfully allow and restrict access to sensitive information.
  If anonymity is granted, reduce the number of people with access to the cleared data.

Data Management

• Clear responsibilities for data management in your organization
• Ensure data are stored securely
  Work with IT to restrict access to data in electronic files through password protection.
  Store hard copies of data in a locked cabinet.
• Set rules
  Decide how long hard copies and electronic data are stored and make sure that data then is destroyed irrecoverable.

Communication

• Be transparent about your measurement tools
• Communicate results
  Inform everyone who was involved in the measurement process about the results of your measurement tools.
• Define lessons learned out of the measurement process and control for change later on

What do you think?

What other rules or standards would you expect in a measurement process, if your work would be measured?

.

References

Smith, P. (1995). On the Unintended Consequences of Publishing Performance Data in the Public Sector. In International Journal of Public Administration, 18(2/3), 277-310.