This section makes five points. Consider the first when your measurement program is going well. Consider the second and third when your performance program is going poorly. Bear the fourth and fifth in mind in either case.

Point 1: The cause and effect of outcomes are not easily established.
Outcomes can, and often do, reveal the impact of the program, but without collaborating data, it is difficult to demonstrate that your program was the cause of the outcome(s). The outcomes of public sector services are inevitably affected by many events outside public control. In the weatherization assistance program, for example, it is not always easy to demonstrate energy savings because the changes introduced to homes may result in changes in the behavior of inhabitants that confounds the analysis. Assume, as a second example, that the goal of energy research is to encourage the development of new technologies that will be adopted by industry and result in energy savings. The outcome may not occur for decades, and while it may be possible to claim that the original research contributed to the final product, it will most likely not be the only contributing factor.

To determine the extent to which a program has affected the outcomes and to measure the impact, you need to do an in-depth analysis. Special program evaluations provide estimates of program impacts and help determine why some programs succeed and other do not. The cost of special program evaluations to demonstrate the causes and effects may outweigh the benefits of knowing more about causal relationships.

Though most benefits are expected to be related to your efforts and the original program plan, others may be viewed as serendipitous impacts. Such unplanned outcomes contribute to the value of programs, and should be reflected in performance results appropriately.

Point 2: Poor results do not necessarily point to poor execution.
If performance objectives are not being met, it is obvious that something is wrong, but performance information itself does not always provide the reason. Instead, it raises a flag requiring investigation. Possibilities include performance expectations that were unrealistic or changed work priorities. Your organization should be able to explain performance results and to define and address the contributing factors.

Point 3: Numerical quotas do not fix defective processes.
There is also a danger when performance objectives become numerical quotas. The setting of numerical goals and quotas does nothing to accomplish improvements in the process. Identify the challenges and changing the processes are what is needed to improve performance and achieve desired outcomes.

Point 4: Measurements only approximate the actual system.
Performance measurement provides a valuable tool for management and continuous improvement. However, people might try to "game" the system in a way that will make their programs look good. Additionally, accurate data may not be available. These are among the reasons why you need to recognize the fact that the measured system is not the same as the actual system.

Point 5: Performance measures do not ensure compliance with laws and regulations.
Performance measures help form the basis for sound performance-based management. Performance measures do not however provide information on adherence to laws and regulations or the effectiveness of internal controls. Bypassing internal controls or noncompliance with laws and regulations may expedite operations and thus result in a "favorable performance" statistic which does not necessarily indicate good performance. For example, a building could be constructed more quickly if safety controls and funding limitations were ignored. Because compliance and internal controls often have a direct effect on performance, care should be taken to supplement performance measurement with other oversight activities to ensure that controls are in place and working as intended and that activities are adhering to laws and regulations.

Copyright 2005 Oak Ridge Associated Universities