Agency-Based Evaluation Proposal

Agency Context 

The evaluation proposal is for the Framingham Public Schools 21 st Century Community Learning Centers (CCLC). The agency offers an after-school program that serves learners who just left school and are getting ready for high school or college. The agency serves the middle school students from Walsh and aims to provide continued and additional support for programs that enrich individuals academically during the period they are not at school. The program supports social and emotional learning for individual students. It also increases the learners’ engagement in their classwork and during lessons and closes the proficiency and opportunity gaps. At the end of the program, the individual students end up well prepared for high school and college. 

Program Context 

The program that has been proposed for evaluation is the after-school program which takes place when the learners are not at school. These programs happen during the summer holidays. The program serves the referred students from Walsh middle school. The program addresses problems such as students’ lack of engagement while they are in the process of learning, the emotional and social problems students face while learning and the proficiency gaps among the students. The program also ensures that the learners a ready to succeed in high school and college. 

The goal of the program is to ensure that by the end of it, the learner will be well prepared for high school and college. Another goal is that by the end of the program, the learners will have increased their proficiency and opportunities. It is also a goal of the program to ensure that in the end, the learner will be emotionally and socially equipped. Another goal is that by the end of the program student will be more engaged in their learning process and their lives too. 

Evaluation Purpose and Significance 

The United States department of education had requested an evaluation that sought to determine whether the agency was viable for a grant. The evaluation was conducted by the Massachusetts Department of Elementary and Secondary Education (DESE). The report from the evaluation indicated that the program carried out by the agency depended on governmental grants to run efficiently. Some evaluations are currently being conducted in the agency. These evaluations aim to discover the learners from the Cameron and Fuller middle school who are eligible for the program. The teachers from Cameron and Fuller middle school are the ones conducting the evaluation, therefore, refer the eligible learners to the CCLC program. 

The evaluation is conducted by assessing the learners, identifying the ones with limitations, and thereafter referring them to the after-school program. To collect the learner’s information, the teachers use their performance results. The evaluations that are currently in the planning phase are the ones meant to ascertain the eligibility of the learners in Fuller and Cameron middle school. The staff at the agency have several evaluation questions that they would like to be answered. This includes how they will be evaluated to identify their roles in helping the learners meet their needs. The staff is also curious about the factors that will be assessed on the learners at the end of the program to prove their contribution to the enhancement of the learner’s wellbeing. 

Practice-Based Importance 

However, the aspect of the program that needs evaluation is cultural diversity. The program does not fully address the diversity in the cultures of the clients subjecting them to experiencing difficulties in matters about the area. Therefore, the program needs to undergo evaluation on the strategies that have been implemented to cater to the cultural diversity of the staff and the learners. Needs assessment is the most recommendable method of determining the needs of both the staff and the learners in the agency. In this case, there is an imminent need for the agency to address the issue of cultural diversity among the staff and the students. 

Agency Link 

Examples of cultural diversity in the agency and how they impact the learning process of the students in the agency are economic, race, religion, and language (Konan et al., 2010). The learners in the agency all come from different backgrounds which differ in their economic status. Economic pressure on a learner can affect their learning process because the ones from a low-income family need to constantly struggle to keep up with the rest to avoid being mocked by the rest. It is also important for the program to recognize the presence of students from different races and address the issue of how their skin color affects the perception that students have of each other. It is also important for the program to address the issue of different requirements of religious traditions and how they impact the learning of the students. Language barriers are an issue that the program needs to address to ensure that students are not demoralized due to their different pronunciations or their lack of knowledge of certain languages. 

The steps to be taken in the evaluation process are first clarifying what is to be evaluated and in this scenario is the issue of cultural diversity. The next step is engaging the stakeholders that are Framingham Public Schools, Massachusetts Department of Elementary and Secondary Education (DESE), and Walsh Middle School. The next step is assessing the evaluability of the issue at hand and the resources needed. The next step is coming up with the evaluation questions such as how are context and language differences addressed when they arise during the learning process. The next step is determining the most appropriate measurement methods and developing an evaluation plan. The final step is collecting, processing, and analyzing data. The expected outcome for the evaluation is that the program will address the aspect of cultural diversity. 

Evidence-Based Importance 

According to Gay (2013), all teachers need to emulate the characteristics of culturally responsive teaching. To emulate these attributes, the teachers need to restructure their beliefs and attitudes about ethnic, racial diversity, and cultural differences. The teacher needs to also resist their resistance to diversity in the cultures in the classrooms. The teachers need to also center their teaching strategies around cultural diversity this happens when they inaugurate connections that are perceived as pedagogical between the lessons taught in the classrooms and a culturally responsive teaching strategy. 

The literature is a review on cultural diversity in the classroom and the implementation of culturally responsive teaching which is a strategy needed in Framingham Public Schools 21 st Century Community Learning Centers (CCLC). The literature has addressed the issues of race, ethnicity, and cultural differences. It highlights how these issues can hinder the learning process of a student. The literature also suggests that teachers should implement a culturally responsive teaching strategy to solve the issue of cultural diversity, therefore, ensuring both the teacher and the learner achieve their goals. The literature is of use to the evidence-based evaluation proposal since it highlights the issue and also gives several methods that could be implemented to solve the issues. It is also important because the basis of the evaluation is how the program addresses the issues that arise rises from the cultural diversity of the learners during the learning process. 

Evaluation Question 

How is cultural diversity addressed when it arises during the process of learning? 

Evaluation Designs and Methods 

Evaluation Approach 

The type of evaluation being conducted on Framingham Public Schools 21 st Century Community Learning Centers (CCLC) is an outcome evaluation. This is because the evaluation aims to study the outcome of addressing cultural diversity among the learners during the learning process and the impact it will have on their overall performance and achievement of goals. The goal of the research is explanatory since it aims to describe why addressing cultural diversity is important to ensure that the learners do not face difficulties in the learning process. The research design is pre-experimental. The type of information needed is both qualitative and quantitative. 

Sampling 

The sampling strategy to be used in the agency-based evaluation proposal is the non-probability strategy. The type of sampling to be used is convenience sampling. The sampling strategy is the most efficient to address the issues on economic, race, religious, and language differences in the program. This is because it enables one to choose specific people who are culturally diverse and use them for sampling to get accurate results. The intended sample population is a group of learners who are between the age of eleven to the age of thirteen who are in the after-school program and are receiving care on emotional and social learning. The sample population incorporates the learners who have been in the program for at least a month. The teacher is the target caregiver. The anticipated sample size is twenty learners who are all culturally diverse from each other. This will allow for the collection of adequate information. The participants will be recruited by selecting the learners that are from different cultural backgrounds. 

Measurement 

Framingham Public Schools 21 st Century Community Learning Centers (CCLC) is an agency that accommodates students with various cultural diversity. This cultural diversity can be categorized as economic, race, religion, and language. These categories form the major content areas of data collection. The kind of data to be collected from the sample population is how cultural diversity is addressed when it arises during the process of learning. The data will inquire information from the learners on the experience they had in an incident caused by their cultural diversity. For example, in the case scenario of a student whose pronunciation differs from the rest due to their language, the data collected should entail information on how the rest of the learners react to the situation and what the teacher did to respond to it. The data collected should address how the economic backgrounds, race, and religions of different learners affect their learning process in the program and what the teachers are doing to ensure that they accommodate all the students in their teaching strategies. 

Measurement Instruments 

Questionnaire 

Framingham Public Schools 21 st Century Community Learning Centers (CCLC) 

Agency–Based Evaluation Proposal 

Questionnaire 

Name……………………………… Age………………………………………………… 

Length of Program Involvement…………………………………………. 

Type of care……………………………………………………. 

At the program we:	Strongly agree	agree	Disagree	Strongly disagree
Set clear expectations in our charter for celebrating the cultural diversities of our learners
We recognized the economic diversity within our learners
Recognize the racial diversity within our learners
We recognize the religious diversity within our learners
Respect cultural protocols of our learners
Build a curriculum that is culturally inclusive to all learners
Use cultural responsive teaching

The rationale behind the format of the questionnaires is to understand what the learners think of the program and how cultural diversity has been addressed by the school. This will help identify where the problem is and therefore make it easier to find and apply the most relevant solutions. 

Data Collection 

The method of data collection is a mailed questionnaire and a face-to-face interview. The questionnaire is meant to get a rough idea of what the students think of how the program addresses racial diversity during the learning process. The face-to-face interview is important to give the students a platform to narrate their experiences of situations where they felt like their cultural diversity was not addressed in the right manner. These methods of data collection are effective and will provide all the information needed for evaluation. 

Data Collection Feasibility 

The feasibility of using mailed questionnaires is that they are not expensive therefore it is a very affordable method of acquiring information. Questionnaires are also very practical in that one can choose the kind of questions to ask and in what format which helps in gathering vast information about any topic ( Queirós, 2017) . Mailed questionnaires are quick and results can be achieved within a short time. Questionnaires have scalability where they allow people to get information from a large audience at a minimum cost. The format of the questionnaire used is easy to analyze and visualize since the answers are straight and to the point. 

One of the disadvantages of using questionnaires is that the learners may interpret the questions differently, therefore, give inaccurate information. Questionnaires also do not convey the emotions and the feelings of the interviewee ( Queirós, 2017) . Some of the questions in the questionnaire may be difficult to answer for some of the learners who end up leaving blank spaces, give answers that are not honest or unconscientious responses 

The feasibility of using a face-to-face interview is that it gives the interviewer a better rate of response in that the interviewee is given a platform to give a deeper explanation of the information provided in the questionnaire (Adhabi & Anozie, 2017) . This way, the interviewer I in a position to tell why the learners chose a certain answer in the questionnaire. Using a face-to-face interview, the interviewer can evaluate the non-verbal behavior of the learner. Face-to-face interviews also allow the interviewer to ask as many questions as they want and also help them clarify what they do not understand. One of the disadvantages of face-to-face interviews is that it is expensive and time-consuming. This is because a lot of time is spent in the interview room gathering adequate information. During a face-to-face interview, the interviewer is in a position to influence the responses of the learners therefore the information gathered may not be accurate. 

Therefore, using both questionnaires and face-to-face interviews ensures that the information obtained is accurate (Adhabi & Anozie, 2017) . This is because the interviewer uses the answers that the learners gave in the questionnaires to conduct a face-to-face interview. Consequently, there is no room for changing the learner’s perspective, and also the learners can clarify the questions that they did not understand. Therefore, making the data collected adequate. 

Data Analysis Plan 

The data collected from both the questionnaire and the face-to-face interview will help analyze the perspective of the learners on whether cultural diversity is addressed in the program. This will be done by analyzing their answers in the questionnaires and listening to their experiences from the face-to-face interview. This information will help the areas that need to be addressed and the learner’s expectations. This will help identify how teachers in the program can apply culturally responsive teaching strategies to ensure that they accommodate all the cultural diversity of their learners while teaching. 

Ethics 

The ethical issues that have been considered in the design and implementation of the evaluation are informed consent where everyone who will be involved in the evaluation will be fully informed of all the relevant details about the evaluation. The participants will be aware of the purpose of the evaluation and the methods of data collection. This will be to ensure that the members of the sample population have given their consent to be used in the evaluation process. Another ethical issue is voluntary participation where all the learners that will participate in the process will be free from coercion and will be there by their own will. The process of data collection will also be conducted in a way that no psychological harm will be inflicted on the participants. Only the coordinator of the program will have access to any information concerning the whole process to ensure confidentiality. 

To enhance confidentiality, the data collection will be anonymous in that the participant’s identity will remain unknown to the public. Concerning the issue of language diversity will be addressed by ensuring that the learners involved are assisted to read or pronounce words that are difficult to them. For religious diversity, the interviewer will ensure that no student will be forced to address an issue or do something that is not allowed in the religious practices. The questionnaires will also use a language that is not diminishing to a certain race due to the racial diversity. To secure informed consent, the program coordinator will ensure that the participant signs a consent form before the evaluation process. 

Feasibility and Translation 

The potential barriers and facilitators towards the proposed evaluation include the lack of enough staff members to support the evaluation activities and data collection. Another potential barrier is that the process of evaluation may be time-consuming and there is not enough time t set aside for the activity. There is also inadequate data that is necessary for the evaluation process to be a success. The evaluation process needs funding to which may break the program’s budget. The facilitators to the proposed evaluation are involving non-locals who will help in data collection and the evaluation activities. Another facilitator is getting connected with the faculty of the program who are in a position to hire individuals that will support the process of evaluation. 

The findings will be presented to the agency’s stakeholders during a meeting set after the evaluation process. The learners are the ones who will gain from the consequences of the findings. Conducting the evaluation will facilitate the development of research and practices within the organization by helping the staff in the agency improve their effectiveness and efficiency in their teaching strategies. It will also help facilitate support for any other programs in the agency that arise from the evaluation and any outreach activities. The evaluation will inform policies of cultural diversity in the agency by identifying the various ways in which the organization does not address cultural diversity in their program and how they can apply culturally responsive teaching strategies. The findings also inform on the perspective that the learners have regarding cultural diversity that arises during the learning process in the program. Through evaluating the different experiences of the learners, the findings inform how the teachers respond to cultural diversity when it is brought up during learning and what needs to be changed. The evaluation will enhance the learning process of the students and ensure that they meet their goals. 

References 

Adhabi, E., & Anozie, C. B. (2017). Literature review for the type of interview in qualitative research.  International Journal of Education ,  9 (3), 86-97. 

Gay, G. (2013). Teaching to and through cultural diversity.  CurricuInquiryuiry ,  43 (1), 48-70. 

Konan, P. N. D., Chatard, A., Selimbegović, L., & Mugny, G. (2010). Cultural diversity in the classroom and its effects on academic performance.  Social Psychology . 

Queirós, A., Faria, D., & Almeida, F. (2017). Strengths and limitations of qualitative and quantitative research methods.  European Journal of Education Studies . 

Appendices 

Program Goals, Activities, and Outcomes (Note: You are required to fill this section out, regardless of whether or not your program has a formal articulation of its goals, objectives, and activities ): 

Goals (i.e., broadly speaking, what does the program want clients to achieve in the future?) 

By the end of the program the student will be ready for high school and college 

The student will be proficient and have increased their opportunities 

The clients will be socially and emotionally equipped by the end of the program 

Students will be significantly engaged in their learning and lives 

Program Activities (i.e., actions the program takes with clients to help them achieve outcomes) 

-The program implements innovative teaching and learning strategies 

-Adopts well-rounded programming which includes, media technology, project-based learning, arts and music, health and wellness, Science Technology Engineering and Mathematics (STEM), as well as other academic enrichment programming which addresses student needs and interests. 

The program provides activities, which are not accessible to the learners and expands their access to high-quality services in the community. 

-Supports the development of support systems and programs, which are culturally relevant and leverages the strength, knowledge, and the students’ educators’ community’s and families’ asserts. 

-It uses effective family engagement strategies, which are culturally collaborative and shows an understanding of the various norms, languages, and values. 

Client Outcomes (i.e., the changes in knowledge, behaviors, attitudes, and/or states of being that will occur for clients as a result of participation in the program) 

Short term	Intermediate	Long-term (if applicable)
The student will have a well-developed relationship with peers	The student will be involving emotions in the learning process	The student will be emotionally and socially equipped
The student will be engaged in the learning process through group works	The student will be significantly creative	The student will be ready for high school and college
The student will be engaged in the learning process through question-and-answer techniques	The student will be collaborative in the learning process	The learner will be proficient to the extent of closing all opportunity gaps
The student will have a well-built relationship with adults	The student will embrace cultural differences as they acquire their knowledge and skills	The learner will be integrating the English Language Arts (ELA) skills in their learning

Consent Form 

I …………………………………… hereby agree to participate in the research project entitled 

(Name of the participant) 

……………………………………………………………… . 

(Name of the agency) 

My participation in the project is voluntary. The purpose of the projects has been clearly explained to me and am therefore my participation is fully consented to. 

………………………………… . ……………………………………… 

(Signature of the participant) (Date)