The reason for carrying out an evaluation is not to prove but rather to improve. Evaluation should not just be a one-time process but an ongoing undertaking that will aid in bettering the decision-making abilities of the program founders and the participants (Lobo, Petrich & Burns, 2014). However, for mots program founders, they view evaluation as being an add-on activity after the program has been completed. In order to determine the effectiveness of the intervention, an outcome-based evaluation was employed. An outcome evaluation helps in ascertaining the effectiveness of program towards attaining its set objectives by measuring the results. It is important for program founders to first outline the desired objectives before launching a program. An outcome-based evaluation will help in determining the differences made by the intervention. The changes in the lives of the participants because of the intervention is what forms the results used in evaluating the program’s effectiveness.
The major advantages of an outcome-based evaluation are that it will help in the improvement of the intervention and services, allows for the attraction of new participants and helps in determining the intervention’s cost-effectiveness. Moreover, this type of evaluation allows for accountability, and this makes it easy to retain and sustain the funding. There are four different levels of outcomes measured by this type of evaluation; reactions, immediate learning outcomes, intermediate behavioral outcomes and long-term outcomes. Nonetheless, this type of evaluation has its fair share of setbacks such that it helps in showing contribution rather than attribution and it is not formal research.
Delegate your assignment to our experts and they will do the rest.
Evaluation Questions
Is the program increasing the self-esteem of the respondents?
Are the respondents feeling any sense of empowerment by being actively involved in the program?
Does the program improve the self-efficacy of the participants?
How many policy changes in Pennsylvania can be attributed to the program?
What percentage of participants report being “satisfied” or “very satisfied” with the program?
What percentage of participants “agree” or “strongly agree” that mental health program is appropriate for the society?
Has the number of participants increased, declined or remained stagnant since the formation of the program?
Are the program’s activities sustainable?
Did the intervention’s activities influence the targeted short-term outcomes?
Did the intervention’s activities influence the targeted long-term outcomes
Is there any base for evidence for the program’s activities?
Evaluation Design
An evaluation design helps in the planning of what to measure and how it should be measured. The evaluation design will lay down a plan that will help in measuring the structures, processes and outcomes of a program. The evaluation design will also determine the most appropriate measurement tools what population should be measured, and the time the measures should be carried out. In this case, the evaluation aims at ascertaining whether the undertakings of the program, have an impact on the respective outcomes. In this case, the evaluation design should be stronger, and this means that it is costly.
The most appropriate evaluation design for this program will be the experimental design. In this type of design, the participants are assigned to the intervention group consisting of those affiliated to the program and the control group consisting of participants who do not become active participant in the program (Thompson & Panacek, 2006). During the assignment of the participants to the intervention and control group, a random selection is used. Random sampling allows every individual to have an equal chance of being a participant in each group and thus eliminating any room for bias (Elfil & Negida, 2017). Biases affect the validity of any research results and findings, and thus the information cannot be used in ascertaining the effectiveness of the intervention. A comparison is carried out between the participants from the intervention and control group. Differences between the two groups are determined. The procedures used in selecting participants from both groups must be similar. The control and intervention groups are measured at two different points and a comparison of the changes over time are later determined. The experimental evaluation design provides one with the strongest evidence on the effectiveness of the intervention program. The only setback is that it is quite expensive and its application in real-world setting can at times be impractical.
Data Collection Plan
During the implementation of a data collection plan, one needs to take into consideration the processes, outcomes and structures data collection.
Structures
The structures form the resources that allow for the effective operation of the intervention program. In most cases, the technique used in collecting structure-related data is non-numerical methods. The main data collection methodology for this will be interviews of the key informants. The key informant form the people who were actively involved in the formation of the intervention program. These people have detailed knowledge about the group, as they are involved in the administration of the program. To carry out the interview, one must first identify the key informants. The interview of the key informants should be individually to ensure that one get more information from people who are reluctant to speak in-group settings.
Process
According to Diaz et al. (2014), the process includes the service delivery abilities of the intervention. Data related to processes can be either qualitative or quantitative. In this case, the data on process should be collected from the participants, and this will involve the use of intervention and control groups. In order to gather this type of data, survey will be used, and they will include open-ended and close-ended questions. Close-ended questions allow one to get data that is easy to analyze while open-ended questions allow the participants to provide more detailed information about a particular question or item (Singer & Couper, 2017). The process questions will be used to determine whether the intervention program was implemented as intended
Outcome
Data on the intervention program outcomes should be quantitative. A survey shall be used in gathering the data on the program outcomes. The survey shall consist of close-ended and open-ended questions. The main aim for a majority of programs is to bring about changes in the behaviors, skills, attitude and knowledge of the interest group. Diaz et al.(2014) state that the survey will be used in determining whether the program outcomes are as it was intended and the results used in ascertaining the effectiveness of the intervention.
Resources Summary
For a program to run effectively, it must have resources. The resources include financial, human and materials. The program will require $100,000. This money will be used in the purchasing of hard paper and paying for the printing and typesetting of the pamphlets that will be distributed across Pennsylvania. The funds will also be used in remunerating the various program personnel and catering for travelling expenses. The program will also require a social media webpage, which will be used in raising awareness on the mental health issues in Pennsylvania. Human resources will include a program coordinator whose duties will comprise working with other staff members to ensure that the intervention is a success. An accountant will also be required to ensure that the funds received from donors and sponsors are effectively used in managing the program activities. In order to ensure that the program works within given laws and regulations a lawyer will also be required. The program will also require a social media expert who will manage the social media page and ensure there is continuous flow of information and response to any questions raised by the public on the social media webpage. The program will also have various mental health professionals such as psychiatrists, psychologist and counsellors who will provide mental health information to the participants.
References
Diaz, T., Guenther, T., Oliphant, N. P., & Muñiz, M. (2014). A proposed model to conduct process and outcome evaluations and implementation research of child health programs in Africa using integrated community case management as an example. Journal of Global Health , 4 (2).
Elfil, M., & Negida, A. (2017). Sampling methods in clinical research; an educational review. Emergency , 5 (1).
Gross, M., Mediavilla, C., & Walter, V. A. (2016). Five steps of outcome-based planning and evaluation for public libraries. Chicago
Lobo, R., Petrich, M., & Burns, S. K. (2014). Supporting health promotion practitioners to undertake evaluation for program development. BMC Public Health , 14 (1), 1315.
Thompson, C. B., & Panacek, E. A. (2006). Research study designs: experimental and quasi-experimental. Air Medical Journal , 25 (6), 242-246.
Singer, E., & Couper, M. P. (2017). Some methodological uses of responses to open questions and other verbatim comments in quantitative surveys. Methods, data, analyses: a journal for quantitative methods and survey methodology (mda) , 11 (2), 115-134.
