Chicago State University Ghana Teaching and Learning Materials Program:

Appendix G: CSU TLMP Ghana TOT Report Chicago State University Ghana Teaching and Learning Materials Program: Training of Trainers Workshop May 2011 Rev. Sep. 2011

Table of Contents Page Executive Summary 3 Introduction...4 Culture of CSU TLMP Ghana...6 Data Collection and Analysis 7 Results...8 Conclusion 19

Executive Summary The Teaching and Learning Materials Program (TLMP) was established under a cooperative agreement between the U.S. Agency for International Development (USAID), Chicago State University (CSU), and the Ghanaian Education Service (GES). In 2006, TLMP program officials developed, printed, and distributed over 2.8 million colored workbooks, Teacher s Guides and Colored Wall Charts also called teaching and learning materials (TLMs), in three subject areas numeracy, literacy, and environmental studies for kindergarten (KG) levels 1 and 2. In September 2008, two national Training of Trainers (TOT) events, sponsored by the CSU TLMP, were held in Ghana, one in Sunyani in the Brong-Afo region and the other in Koforidua in the Eastern region. Trainers acquired specific knowledge and skills to train 300 kindergarten teachers in 14 districts using activities and pedagogy related to early childhood education. The fidelity of the training in the districts is unknown. A 2010 assessment showed that teaching practices at the classroom level had changed in regards to the use of TLMs. According to the cooperative agreement and recent modifications, 2,000 additional teachers will be trained under the program s teacher training component by 2012. The key objectives of the teacher training strategy are threefold: (a) to enhance teachers expertise in using the TLMP materials; (b) to equip them with a deeper understanding of the best practices for use of the materials and (c) to prepare participants to use the train-the-trainer model to disseminate best practices to teachers. The design phase for the second installation of the TOT began in 2010. With the TOT model as a delivery mechanism, knowledge and skills will flow to two levels of the education system: 1. Administrative: CSU TLMP Ghana (facilitators) will train district-level trainers, including early childhood coordinators (ECCs) and training officers (TOs). 2. Classroom: ECCs and TOs will train kindergarten teachers. The author of this report used Guskey s professional development evaluation model to evaluate the pilot workshop implemented for the first tier of the TOT model in May 2011. The study was conducted to determine whether the training was effective when the facilitators directly trained this new cohort of 24 members. This report is provided to key stakeholders at CSU in order to make informed decisions regarding teacher training, use of the TOT model, and professional development.

Introduction The CSU TLMP team has designed teaching and learning materials in three subject areas numeracy, literacy, and environmental studies for KG levels 1 and 2 in all Ghanaian school districts on a nationwide scale. The program provided technical support, development, production, and distribution of high-quality, cost-effective textbooks and learning materials as part of its initial goal of providing curriculum resources to primary classrooms by 2012. During the initial funding cycle (2006 to 2008) curriculum materials were produced and distributed over 2,171,000 units of teaching and learning materials, serving 1.6 million pupils and 22,000 teachers annually. The next funding cycle (2008 to 2012), production expanded to a national scale covering 124 districts. Per the cooperative agreement, the program includes a teacher training component to train teachers in the use of TLMs, Peer learning, and participatory teaching techniques. These three topics are integrated into the program s child-centered teaching methods. Childcentered teaching is an approach that actively engages students in the construction of knowledge and their own learning. The TOT workshop topics incorporate these same methods in their instruction, print materials, and resources made available to workshop participants. CSU TLMP team created and designed five training modules based on components of the 2008 TOT workshop, The modules were created in response to request made by the Ghanian team with a special focus on the following areas: How Children Learn; Learning Styles, Teaching Methods, Teaching Strategies, and Songs, Poems. The cohort of districts that took part in the implementation of the teacher training cascade included Accra Metro, Tema Metro, Ga East and West Municipal, Kokuku Krowor Municipal, Akwapen North, Birim North, Kwahu North, New Juaben, and Suhum Kraboa Coaltar from the Greater Accra and Eastern regions. The teacher training objectives were designed to be accomplished at two levels. The first level of the design was organized around the training of district-level trainers (TOs and ECCs). Program facilitators (both Ghanian and CSU) introduced and modeled key concepts to district-level trainers. In total the facilitators spent 40 hours, distributed over 5 days, providing instruction on theoretical underpinnings, content knowledge, and pedagological skills related to the KG1 and KG2 population. District level trainers then spent several hours each evening studying information imparted during the day, and completing assignments designed to enhance their leaning and skills. About 45% of the training time was spent on dissemination of information contained in the CSU TLMP Ghana teaching and learning materials. In contrast, about 14% of the time was allotted to preparation of TLMs, and 8% went toward development of training skills. The rest of the time was spent on issues that directly or indirectly contributed to learning and skill development, such as recap of lessons, customary ceremonies, and group assignments. Training objectives will be met at the second level through implementation of training by district level trainers. District-level personnel will facilitate TOT workshops in their respective districts to help KG1 and KG2 teachers master the CSU TLMP curriculum

materials; teaching approaches and enhances their pedagogical and professional skills. The TOT model will help personnel to monitor implementation of training, professional development of teachers, child outcomes and, most importantly, continuously assess the impact of the training. During the training, the TOs and ECCs were asked to develop plans for the implementation of the TOT workshop in their district. The TOT training manual included an evaluation trainers could give to trainees at the next level to assess the effectiveness of the workshop. The trainers will return this information to the CSU TLMP Ghana office. This report is organized into several sections. First, the data collection and analysis plan summarizes the data sources, analysis, and evaluation questions. Results of the workshop are then presented, based on the corresponding level of Guskeys model followed by an analysis of the impact. The final section includes the recommendations and conclusions.

Culture of CSU TLMP Ghana Key words and phases were identified to accurately describe the values of the CSU TLMP Ghana training. The core values were decision making, facilitation, leadership, calendar, and time. Two core beliefs were found in the CSU TLMP Ghana TOT training: (a) the importance of resources and (b) collaboration between partners and participants. These core beliefs helped participating members to bond during the training. The beliefs also had characteristics in common with the culture s values. For example, CSU TLMP Ghana officials made decisions to delegate specific tasks and identified resource persons to lead modules; based on their field of expertise. The dominant themes within the participants beliefs and values spoke to the culture s need to work with limited resources and encourage cooperation. Rituals embedded in the TOT workshop included such behaviors as arrivals and greetings, meals, communication during the workshop, and daily dismissal. Observing the participants eyes on the speaker or facilitator and the extent to which their questions and comments related to the subject matter being presented and/or discussed assessed attention to these ritualistic behaviors. Attentive engagement was evident at all meetings, regardless of the facilitator. Silence and respect were always accorded to presenters; participants appeared to know appropriate times for response and feedback. These and other rituals such as singing nursery rhymes native to the Ghanian culture, opening and closing ceremonies, daily prayers, and recap of the previous day s activities placed a certain level of importance on partnership and being a culture that was respectful, and proactive. Study Participants Comments During the semistructured interviews, participants answered questions about the training. Their comments related to the section that follows the summary.

Table 1. Data Collection and Analysis Plan for Guskey s Model Training evaluation questions/tot objectives Training Evaluation Question 1: To what extent did district-level trainers participation in the CSU TLMP address the five levels of professional development according to Guskey s model? Subquestion 1 (Guskey Level 1): What were the teacher trainer s reactions to the training? Subquestion 2 (Guskey Level 2): To what extent did the teacher trainers learn the intended materials? TOT Objective 2: To prepare participants to use the train-the-trainer model to disseminate the best practices learned in training to teachers. Subquestion 5 (Guskey Level 5): To what extent did learning outcomes change? Methods of data collection Professional development training evaluation Semistructured interviews with participants Conducting observations of TOT model implementation and TLMP sessions Methods of data analysis Statistical analysis to determine means NVivo software to identify themes and their frequency Subquestion 3 (Guskey Level 3): To what extent did the organization support the teacher training event? Subquestion 4 (Guskey Level 4): To what extent did the teacher trainers demonstrate skills learned in the program? TOT Objective 1: To enhance teachers expertise in using the TLMP materials and equip them with a deeper understanding of the best practices for use of the materials Professional development Statistical analysis to training evaluation determine means Semistructured interviews NVivo software to with participants identify themes and Conducting observations of their frequency TOT model implementation Annual planning and and TLMP sessions budget review; submission of objectives University support planning and budgeting documents and accomplishments Professional development training evaluation Conducting observations of TOT model implementation and TLMP sessions Statistical analysis to determine means Training Evaluation Question 2: To what extent did the background, context, structure, and process (learning and impact) features influence the professional development experience? Training Evaluation Question 3: According to the teacher trainers, which features of the program were most significant in their professional development? Professional development training evaluation Professional training evaluation Statistical analysis to determine means and frequencies (percentages) NVivo software identified themes and their frequency

8 Results Guskey s professional development evaluation model comprises five critical levels, which were used to examine the effectiveness of the training. The five levels of the evaluation model include (a) participants reactions, (b) participants learning, (c) organization support and change, (d) participants use of knowledge and skills, and (e) student learning outcomes. In this section, the background of the trainers is discussed, as well as an analysis of the data findings for each of Guskey s levels. Trainer Characteristics District-level personnel were compared and contrasted according to a number of characteristics as measured by several surveys. Table 2 shows that the group was mostly uniform in terms of gender. The majority of those who attended holds a bachelor s degree and were experienced teachers at all grade levels. Most trainers had attended more than 10 in-service training events. Table 2. Trainers Background Characteristics and Training Participation Characteristic Percentage of trainers (N = 19) Gender Male 47 Female 53 Professional qualifications Master s 21 Bachelor s 58 Diploma 16 Certificate 5 Years of being a teacher 1 to 2 years 5 10 to 15 years 21 More than 16 years 74 Grades taught Primary 11 Junior high school 21 Primary and junior high school 21 Junior and senior high school 5 Primary through senior high school 16 All education levels 5 Other 21 Number of prior in-service trainings 3 to 5 16 6 to 9 21 more than 10 63 Training Experience Trainers provided information about their experience with the TOT model, their years of experience as trainers, and whether or not it was the first time they attended this 8

9 workshop. Table 3 shows that 38% of the trainers had no previous experience in implementing the TOT model. Most (74%) of the trainers had retained their current positions for 1 5 years. However, the trainers tended to have more experience as trainers than as teachers. As Table 2 shows, 58% of participants had less than 5 years experience as trainers. Table 3. Trainers Previous Training Experience Experience Percentages (N = 19) Experience with training-of-trainer model None 37 1 2 times 32 3 5 times 21 More than 5 times 10 Years in current position 1 2 years 42 3 5 years 32 6 10 years 21 More than 11 years 5 Years of training 1 2 years 47 3 5 years 11 6 10 years 31 More than 11 years 11 First CSU TLMP Ghana training Yes 63 No 26 Missing 11 Most of the trainer survey respondents (63%) had not participated in a previous CSU TLMP Ghana TOT training. Five trainers participated in the 2008 training. Altogether, 40 hours, distributed over 5 days, were spent learning and developing skills in the course. The trainers also spent several hours each evening preparing their lessons. About 45% of the training time was spent on dissemination of information contained in the CSU TLMP Ghana teaching and learning materials. In contrast, about 14% of the time was allotted to preparation of TLMs, and 8% went toward development of training skills. About 33% of the time was spent on issues that directly or indirectly contributed to learning and skill development, such as recap of lessons, customary ceremonies, and group assignments. Guskey s Level 1: What were the teacher trainers reactions to the training? Survey Items 3. The training covered the topic in sufficient detail. 4. The presentation was effectively delivered and well organized. 5. The venue is suitable for the training. 6. The training created an atmosphere where all or most of the trainers participated. 7. Time was used effectively. 9

10 8. The materials were relevant, useful, and made available during training. Rating scale: 1 = strongly agree, 2 = agree, 3 = neutral/not applicable, 4 = disagree, and 5 = strongly disagree. Number of responses: 19. Summative Information For the CSU TLMP TOT workshop, 94% of participants either agreed or strongly agreed that the topics provided sufficient detail and the presentations were well organized. A consensus of 89% either agreed or strongly agreed that the facilities were suitable. A consensus of 99% of respondents either agreed or strongly agreed that the atmosphere encouraged participation from trainers, compared to 93% for effective use of time. The participants either strongly agreed or agreed that the training materials were relevant, which received the strongest consensus of 100%. Figure 1. Guskey s Level 1 evaluation. Data Use for Improvement The data collected for Items 3 8 indicate the following findings: (a) the training was well organized, (b) the training venue was appropriate, (c) the topics were explained in adequate detail, (d) the atmosphere encouraged trainers to engage and participate, (e) time was used effectively, and (f) the training materials were relevant. During the week-long workshop, this researcher noted that the Train the- Trainer Manual, developed by the CSU TLMP team was not introduced until the final day. The manual was designed to support the professional development of trainers working with future teachers throughout Ghana s public school system. It was distributed on the final day because of technical problems with production. Recommended change for future TOT workshops include: (a) early introduction of the Training Manual to emphasize the trainer s role in the professional development of teachers, and (b) the inclusion of specific periods of reflection during the course of workshop sessions. This will stress the value of reflection and critical thinking, cornerstones of sound educational practice. Comments The workshop was well organized and interesting. Everything was done well. 10

11 I have taken a lot of trainings as they have become available. I think always seeking learning opportunity is so important, especially when it comes to the changes at the KG level. I am very excited about what I learned in this workshop and will try to copy this similar environment. Guskey s Level 2: To what extent did the teacher-trainers learn the intended materials? Survey Items 18a. My understanding of the topics PRIOR to attending the session 18b. My understanding of the topics AFTER attending the session 19a. My ability to demonstrate comprehension of the topics PRIOR to attending the session 19b. My ability to demonstrate comprehension of the topics AFTER attending the session 20a. My ability to apply concepts to both my teaching and training colleagues in this area PRIOR to attending the session 20b. My ability to apply concepts to both my teaching and training colleagues in this area AFTER attending the session Rating scale: 1 = excellent (E), 2 = very good (VG), 3 = good (G), 4 = fair (F), and 5 = poor (P). Number of responses for Item 18 = 16, Item 19 = 15, and Item 20 = 17. Summative Information Trainers received a retrospective pretest at the end of the training to assess their knowledge before and after the session. As seen in Figure 2, the posttest consensus increased by an average of 22%. Figure 2. Retrospective test. Data Use for Improvement The data collected from the retrospective test items verified the achievement of the trainers learning outcomes. All levels of knowledge increased significantly for participants except their ability to demonstrate the topics (73%). This researcher notes that each trainer did receive some materials from CSU TLMP team members on Power Point, and in 11

12 print through the Trainer s Learning Guide distributed during the first day of presentation. However, additional recommendations for consideration include: a) postings on the CSU TLMP website for distribution to the Ghana office, and b) future training sessions include more opportunities for practice, with trainers serving as the facilitators and explaining the various topic areas. Survey Items 1. I understand and can describe my role in the implementation of CSU TLMP Ghana. 2. The objectives of the training were clearly stated. 9. The training activities helped me acquire the intended knowledge and skills. 13. Follow-up and ongoing assistance were offered to help me implement changes advocated in the training. Rating scale: 1 = strongly agree, 2 = agree, 3 = neutral/not applicable, 4 = disagree, and 5 = strongly disagree. Number of responses = 19 Summative Information Trainers were asked how comfortable they felt in their role delivering the workshops to others. Many expressed that they were well prepared to deliver training, that the training they received was good, and that their current GES positions made it easier to train others. Eighty-six percent of the trainers either strongly agreed or agreed that they understood their role in the CSU TLMP Ghana. A consensus of 97% of respondents either strongly agreed or agreed that the objectives of the training were clear. A consensus of 94% of participants either strongly agreed or agreed that the training activities helped them to acquire the intended knowledge and skills. A consensus of 82% of the district trainers either strongly agreed or agreed that the program administrators verbalized the option for follow-up assistance. CSU TLMP contacted participants via email and telephone, after the initial workshop to address any questions remaining about the program. As of September 2011 the long term follow-up strategies were undetermined, and not the focus of this report. Figure 3. Guskey s Level 2 evaluation. 12

13 Data Use for Improvement The data collected indicated the following findings: (a) trainers understood their roles, (b) the objectives were clear, (c) skills and knowledge was acquired through the activities, and (d) follow-up support was offered. No changes were recommended for Items 1, 2, and 9. However, the need for follow-up support is recommended: a) planned post-training checkins provided through field supervisors until the program is terminated; b) follow-up survey administered several months after the trainers have presented the CSU TLMP Ghana TOT workshop to the teachers to help them reflect on the greatest strengths of the training they received; and, 3) interview trainers to solicit feedback, request training-related examples, and identify additional training needs of the participants. Comments A good trainer helps the teacher make connections and also relates the content to relevant standards and practices in the classroom. Engagement and making important connections to the TLMs are vital to the training experience. Some of the information and activities provided may often be common sense or child friendly, which gets the students involved; they just work! We were divided into groups to work out the solution of a particular problem. We realized that it was easier to work together because we could bounce our ideas off of each other. At first, I thought we were being graded, but that wasn t true. The point of the assignment was to demonstrate how we can find a solution ourselves. Guskey s Level 3: To what extent did the organization support the teacher training event? Review of CSU TLMP Policy and Procedures The review of policy and procedures focused on four areas: organizational support, TOT/TLMP structures, monetary support, and technical support. The review process began by examining the link to the program website, located under the A to Z list on the university homepage. The researcher reviewed the materials on the website, specifically focusing on teacher training. An assessment report regarding the TLMP Ghana program was made available to the university. However, the teacher training materials supporting the TOT workshop were not accessible on the website. The researcher was informed that many teachers in Ghana lack access to the Internet; thus, the materials were housed in the home office in Ghana. Additionally, the trainers received a toolkit that included a USB drive, the Train-the-Trainer manual, and hard copies of the materials presented at the workshop. For a donor-funded program, confirmation of monetary support was given in the budget specified in the cooperative agreement for the CSU TLMP Ghana program. CSU documents verified that a designated fiscal account for TLMP exists and will continue to exist until the completion of the project. Line items in the budget included teacher training, books and resources, staff, printing, postage, travel, supplies, and salaries for team members. 13

14 Evidence of organizational support for the teacher training was apparent in the collaborative design of the CSU TLMP Ghana team. The team is documented on the CSU website, along with information about the overall program. Evidence of the TOT TLMP structures is based on correspondence between internal and external stakeholders known as the CSU College of Education, the University of Cape Coast, the University of Education at Winneba, the Public Accounts Committee, CRDD, and USAID. Data Use for Improvement The data indicated that policy and procedures are in place to support the teacher training workshop at the district level and its initial implementation. Guskey s Level 4: To what extent did the teacher-trainers demonstrate skills learned in the program? Survey Items 10. The training used theory and best practices related to using the TLMP materials. 11. The training activities will help me integrate this material into my instruction. 12. This training provided opportunities to practice my new learning. 15. The training provided information on how students learn. Rating scale: 1 = strongly agree, 2 = agree, 3 = neutral/not applicable, 4 = disagree, and 5 = strongly disagree. Number of responses for Items 10 12 and 15 = 19. Summative Information For the professional development items, 94% of respondents either strongly agreed or agreed the training used theory and best practices. A consensus of 96% of participants either strongly agreed or agreed that the activities helped them integrate the materials into their instruction and provided information on student learning. A consensus of 98% of trainers either strongly agreed or agreed that the training provided opportunities to practice. Figure 4. Guskey s Level 4 evaluation. 14

15 Data Use for Improvement The data collected for evaluation of Questions 10 12 and 15 showed that the training used theory and best practices, integration of materials, opportunities to practice new learning, and information on how students learn. Therefore, no recommendations were made about the training in these areas. Comments The purpose of the theoretical part of the workshop was incorporated in our assignment. In the first 10 minutes, the facilitator introduced the concepts in the form of a lecture. She discussed a particular problem and provided different methods to help us work with students. No one told us how to use active learning; we were activated and learning the materials ourselves. I think that, as trainers, we need to know how to impart information effectively, learning styles, etc., and we need to know how to use the TLMS. This will encourage the teachers to use and become familiar with books. The activities by the facilitators are important and set the tone of the training. Guskey s Level 5: To what extent did learning outcomes change? Survey Items 16. Overall, the training had a positive impact on my professional development. 17. Overall, my new learning will have a positive impact on my training sessions. Rating scale: 1 = strongly agree, 2 = agree, 3 = disagree, 4 = strongly disagree, and 5 = not applicable. Number of responses = 19 Summative Information Ninety-nine percent of the respondents either strongly agreed or agreed that the training had a positive impact on their professional development. Ninety-six percent of the participants either strongly agreed or agreed that the training would have a positive impact on their training sessions. 15

16 Figure 5. Guskey s level V evaluation. Data Use for Improvement The data provided evidence of the impact of the training. Therefore, no recommendations were made about the training in these areas. Survey Items 21. Which of the following statements best describes the overall usefulness of the training? Forced-choice scale: (a) It was a good start; (b) It was a good start, but I have a lot of questions; (c) It was a good start, and I look forward to using the new ideas; (d) It provided everything I need to use the new ideas; (e) I don t think that these ideas will work very well; and (f) It s too soon to tell. Number of responses = 19. Summative Information In regards to the usefulness of the training, 15 trainers (79%) stated that it was a good start and that they looked forward to using the new ideas. Three trainers (16%) felt that it provided everything they needed to use the new ideas, and 1 trainer (5%) stated that the training was a good start. Comments I have many years of experience as a trainer to draw upon for insight. I believe that because of my experience, the teachers in my district trust me and look to me for answers when they are lost. This training will help me to encourage and assist my teachers to pursue a higher standard of teaching to best improve their students learning. As a newcomer to this area, the training was very helpful and helped me make a connection with primary education. By using the teaching and learning materials, I can familiarize myself with the specific needs and interests of the kindergarten teacher. Data Use for Improvement 16

17 Data supporting the overall usefulness of the training were collected. Therefore, no recommendations were made to improve training in these areas. Observations: Summative Information Observations of 15 sessions were conducted to examine engagement and facilitators and trainers implementation of techniques learned in training. The observation protocol used the following Likert rating scale: 1 = most times, 2 = sometimes, 3 = few times, 4 = none of the time, and 5 = not applicable. On average, over 93% of the facilitators encouraged engagement from trainers to participate in the workshop. The following facilitator behaviors were observed most frequently: reading from the teacher s guide and student workbooks, explaining information from the TLMs, writing TLMs related questions on the flipchart, asking for TLM-related information, reciting or correcting trainers recitation, demonstrating sample lesson plans, and preparing for the training sessions. The percentage of trainers demonstrating these behaviors most of the time or sometimes was relatively high, in the 80% 100% range. On average, over 95% of the trainers could be observed actively and attentively participating during the sessions by maintaining eye contact with the facilitator, taking notes on the topic at the appropriate time, sitting up in their seats, asking questions, volunteering to participate in a simulation, and/or referring to teacher s guide and student workbooks when appropriate. However, connecting the session to the TOT model and dissemination of best practices occurred an average of 70% of the time during the session observations. Emphasis on the TOT process and the trainers' roles therein was sometimes compromised. For example, facilitators did not, consistently follow-up on student responses missing the opportunity to foster higher-level thinking. Best practices suggest that effective trainers incorporate the following strategies: review, introduction of new content, model/demonstrate, practice, discussion, and review. It is the use of these techniques that lend themselves to effective training. It is recommended that facilitators engage district trainers in meaningful thought and discussion around this process model. Interviews: Summative Information The respondents volunteered to participate in semi-structured interviews to address the five levels of Guskey s model. Trainers, facilitators, and program administrators volunteered to partake in the semi-structured interviews (N = 15). In-person interviews were conducted with the CSU TLMP Ghana workshop attendees in May 2011 and generally lasted about 30 minutes. The most common responses are listed in Table 4. The total number of answers may have exceeded the total number of respondents because some of the answers belonged to more than one emergent theme. The percentages shown after the total responses for each emergent theme were computed using the total number of respondents. The analysis of the data related to the training depicted engagement, group dynamics, effective use of communication, application, and professional development activities as core themes of the training. 17

18 Table 4. Participants Semi-structured Interview Perceptions of the Professional Development Training Participant responses Emergent themes Freq. Percentage Engagement a. Read TLMs 11 5.3 b. Asked questions 8 3.8 b. Attentive/involvement 12 5.7 c. Commitment/enthusiasm 10 4.8 Group dynamics a. Collaborative/working together 10 4.8 b. Sharing 8 3.8 b. Learn from each other 9 4.3 Training a. Training went well 16 7.7 b. Materials resources were provided 14 6.7 c Practice 15 7.2 d. Collaboration 6 2.9 d. Appropriate schedule 3 1.4 e. Facilitator/CSU TLMP Ghana partners active 2 1.0 Application Learned new techniques/strategies 17 8.1 Child-centered education 11 5.3 Demonstrations/hands-on were essential 14 6.7 Activities Learned skills essential to training 11 5.3 Learned something new 7 3.3 Importance of assessment 8 3.8 Make learning fun 4 1.9 Improvements a. Support for trainers 6 2.9 b. Invite kindergarten teachers to workshop 5 2.4 18

19 c. Development of training materials 2 1.0 Total 209 100 Data Use for Improvement Data collected from the observations and semi-structured interviews with participants provided evidence that the training was well received. No changes were recommended in this area. Facilitator-blended interactive learning tools (e.g., discussions, presentations, demonstrations) encouraged trainers to be extremely engaged. This engagement was noted in the following interviewee comments: Everything has gone well, from the teaching protocols to involvement of the participant. It wasn t teacher centered; it was teacher participant oriented. Initially, we didn t have materials to review with them. We were expecting the teachers to review some materials and prepare lessons, which affects standardization. Now we have something to use that is structured, with lessons, keywords, and literacy reinforcement. The trainers have been together since the start and have gotten to know each other to form a team. They work together to complete a task or presentation. They all contributed to create a shared product through the group interactions. Interviewees had additional comments regarding the use of the TOT model in future trainings. Sometimes we train in difficult training environments that are totally opposite from this training site. Funding is always needed. We can travel to the many places, and we need money for gas. Plus we have to bring the new materials and information to the teachers just to make the training work. It will be challenging to train teachers who do not understand the basics and the importance of these basics, such as children s learning styles. In our district, we receive lots of requests for training, and it is hard to travel in the northern regions, especially if it s raining. This is an excellent workshop, but I don t know when we can take it to the next level. It is cost-effective when compared to the number of teachers we must train. Question 2: To what extent did contextual factors, structural features, and process features influence the professional development experience? Survey Items 1. Background: Pretraining test 2. Learning: Content and practice 3. Contextual: Implementation, support, and follow-up 4. Structure: Delivery and active learning 5. Impact: Post-training test and knowledge Table 5. Framework for Evaluating Professional Development Training Surveys 19

20 Item number Professional development features A L Demographic information 1 Implementation 2 4 Content 5 7 Delivery 8 Support and change 9, 11, 17 Active learning 13 Follow-up 12 Practice 10, 14 16 Knowledge 18 20 Pre-training items Retrospective test Post-training items 21 Usefulness Weighted averages were used in this section. Aggregating data across the item on the professional development evaluation gave a more significant and useful average in accurately ranking the categories. Summative Information After the data were analyzed, various features were found to influence the extent of the professional development experience for participants. Learning was the first component, with an average of 22% (M = 95), which was tied to structure (M = 94), followed by contextual (M = 89) and impact (M = 88) factors at 21%. The trainers were familiar with the concepts introduced in the training due to their backgrounds and demographic variables. However, the trainers prior knowledge (M = 60) had the least influence, with an average of 14%. Figure 6. Professional development features. Data Use for Improvement Data indicated that the professional development features were in place in the overall TOT workshop. To further understand the results, it would be useful to know more about these features in the individualized sessions. In response to this need, it is recommended that an 20

21 evaluation be conducted after each module to collect evidence of the elements and to create a collaborative approach to onsite quality improvement. Question 3: According to the trainers, which features of the program were most significant in their professional development? Summative Information Data collected from the professional development evaluations confirmed that the training materials were the most significant to trainers, with a consensus of 100% (M = 1) Recommendations This section provides the overall recommendations based on the findings of this study: Dispense a similar program to other kindergarten teachers in both public and private sectors. An annual refresher session for all District Trainers that occurs in tandem with current GES training. Selection of a core group of District Trainers, who could be trained in higher-level activities such as monitoring and evaluation, to assess workshops and classroom practices in the region. Establish a professional learning community for the District Trainers whereby they can share ideas, address issues, receive feedback about training sessions and continue to learn. Conclusion This section provides conclusions based on an analysis of the quantitative and qualitative data collected in this study. Surveys, interviews, observations, and analysis of program reports indicated that the TOT model of training was effective by nearly every measure. Specific conclusions are as follows: The results clearly showed that the CSU TLMP TOT workshop was effective and attained strong group consensus as indicated through the various Guskey levels. Statistically significant effects occurred with respect to trainers knowledge acquisition in the retrospective test. Trainers felt well prepared to deliver the training. Participants had very few suggestions for improving the training, and many stated that they liked it as currently designed and delivered. The findings could be used as baseline data during full implementation of the teacher training component and could be used in efforts to develop data for future needs assessments. Additional unsolicited comments almost uniformly showed that the trainers thought the program was useful, excellent, and added value to teacher training in early childhood development and education. 21

22 22