The Importance of Accountability in Program Implementation

The need for accountability in the implementation of programs has witnessed a shift from a concentration on the inputs that entities should put in place to a focus on the results (outcomes) that the programs achieve. An excellent example of this shift is the focus of the US Congress in 1994 on measures such as graduation rates and competency in various disciplines to gauge the effectiveness of educational programs rather than concentrating on inputs such as the play space in institutions. The need to measure the outcomes of programs places a unique mandate on evaluators. In order to successfully undertake evaluation studies, evaluators should be aware of the specific measures that they are to assess, the source of the data to be collected, and the ways of collecting the data. Some of the variables that evaluators utilize in their studies include the outcomes of the program, inadvertent consequences, intermediate measures of progress, program processes, and inputs. The measurement of different variables by evaluators is undertaken using different levels of measurement, such as nominal, ordinal, interval, or ratio. Nominal measures utilize the categorization of data. Ordinal measures rank data in ascending or descending order. Interval measures use equal intervals while ratio measures utilize the features of all the other levels and have a scale based on true zero. Ratio and interval measures are easy to analyze and thus preferable to statisticians. Before embarking on the evaluation, evaluators must come to a consensus with the program managers and other stakeholders on the measures to be evaluated and the indicators of the success of the program. This is because the results of the evaluation are helpful to the program managers; thus, the need to involve them. The measurement of the program outcomes can be undertaken on individuals, organizations, or communities e.g., the increase in standards of living amongst community members aided to start businesses. The choice of the measures to determine program outcomes is dependent on factors such as decisional timetable, time available, money, skills, etc. The outcome measures can also include either short term, long term indicators, or a combination. In preparing the evaluation measures, evaluators should anticipate some of the unintended consequences (both desirable and undesirable) and include them as part of the evaluation criteria of the program. Evaluators can use the opinions of the critics of a program to get a view of the unintended consequences. For example, the critics of the welfare program to the indigent in the US in 1990 helped evaluators to include discontinuation of work by the poor as part of the evaluation measures. Evaluators also utilize intermediate measures of progress to assess a program, especially when the outcomes of the program can only be evaluated in the long term. The interim measures help to give preliminary feedback about the success of the implementation of programs to managers and policymakers. The inclusion of program processes as part of the evaluation measures helps to shed light on the operation of the program, whether it is doing well or poorly. Evaluators use accepted performance standards to evaluate the quality of program processes. Program inputs, resources, and environment are also included in evaluation measures. They include years of operation, budgetary allocations, locations, program participants, etc. The inclusion of these measures is crucial as it helps in the description of the program, assessment of the accrued benefits out of the incurred costs, and also in keeping a check on the effects of varying inputs. In programs undertaken in different locations, input variables are crucial in making comparisons. One of the most crucial decisions that evaluators have to make is the measures to include in their studies. They should strive to include multiple measures since a single measure cannot be adequate to carry out the evaluation. However, in choosing multiple measures, they should ensure that leanness efficiency is maintained. It is prudent for evaluators to use already existent instrumentation since it helps to ensure reliability and forms a comparison base with previous programs. The evaluation measures can be located from journals, government documents, computerized libraries databases, hard copies, etc. In the absence of appropriate existing evaluation measures, evaluators should create new evaluation measures that are suitable to their programs. The process is, however, rigorous and time-consuming. It involves the preparation of questionnaires, surveys, and interview forms. The prepared questionnaires and survey interview questions are subjected to a series of preliminary studies, critiques, revisions, and pilot testing until the measures can generate useful data for the study. The pre-existing and newly developed measures of evaluation should possess some features which make them desirable and appropriate for program evaluations. The measures should be valid, meaning they should encapsulate the intended concepts. The measures should also be reliable; they should provide the same results in different attempts to measure the same occurrence. The reliability is measured using Cronbach's alpha that measures internal consistency. The evaluation measure should also have a sense of direction, the movement of the measures such as good or bad, more or less, etc. The measure should also accommodate the differences in data by being sensitive and embracing different data. The measure should also strive to capture current data since past information on opinions and attitudes may be distorted. The outcome measures being evaluated should be realistically connected to the program initiatives. The measures should also strive to ensure that the data is not prejudiced by the data collection techniques and social desirability bias. The measure should also aim to collect data that is available. If data is unavailable, new measures should be created. Finally, the measures should ensure that data is systematically and consistently collected for comparisons and desirability. The adherence to various recommendations when developing measures by evaluators helps to ensure that program evaluations generate useful data. The data can be used by program managers, policymakers, and staff to make crucial decisions on other programs in the future.