Rating 2016 UNC School Counseling Program Evaluation Report 1 THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL SCHOOL OF EDUCATION - SCHOOL COUNSELING, M.ED. 2016 PROGRAM EVAULATION REPORT EXECUTIVE SUMMARY This report summarizes program evaluation data for the 2015-16 UNC Chapel Hill School Counseling Program, obtained through electronic questionnaires completed by five major stakeholder groups: (1) current employers of 2015 graduates; (2) internship site supervisors of current (August 2016 graduates); (3) alumni of the 2015 graduating cohort; current students (August 2016); and (5) adjunct and non-core faculty who taught in the program in 2015-16. Overall, alumni, employers, current students, and site supervisors hold high perceptions of our program, as demonstrated in the following table (1 = poor; 2 = fair; 3 = good; 4 = excellent): Overall Perception of the UNC School Counseling Program 4 3 2 1 Alumni Employers Site supervisors Students Stakeholder Groups Graduates of the UNC School Counseling program continue to meet the academic, career, and social/emotional needs of K-12 students. Of the 25 graduates in 2016, 88% had committed to school counseling or other education-related positions even prior to their official graduation. Across stakeholder groups, participants in the program evaluation process consistently give UNC-Chapel Hill School Counseling students the highest ratings in their understanding of the school counseling profession and ethical practice, and their ability to conduct group and individual counseling. The areas rated lowest among stakeholders included students abilities to effectively administer and interpret assessments and to conduct research to evaluate the school counseling program. Graduates and students appreciate the cohort model; the support of the program faculty; the program s strengths-based framework; and focus on the ASCA National Model. Employers and supervisors note the high level of preparedness of the program s interns and graduates to quickly assume the role of the school counselor. Stakeholders suggest that the program should work to incorporate more student training on the use of relevant software used to facilitate the school counseling program. The program should provide distance-based or alternative modes of delivering training, including professional development, to site supervisors and professional counselors. Additionally, the program should consider re-sequencing some of the courses and providing students with more exposure to mental health information and additional theories and techniques, specifically for application with young children. Data from this report will be used to make quality changes and additions in areas such as curriculum sequencing, course content, site supervisor training, and professional development for school counselors.
Rating 2016 UNC School Counseling Program Evaluation Report 2 THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL SCHOOL OF EDUCATION SCHOOL COUNSELING, M.ED. 2016 EVAULATION REPORT Meghan Walter, Ph.D. July 2016 This report summarizes program evaluation data collected during the 2015-2016 academic year. Surveys were sent electronically to the following five stakeholder groups: recent (2015) graduates of the program; employers of recent (2015) graduates; site supervisors of current students; current students (2016 graduates); and adjunct/non-core faculty who taught in the school counseling program during the 2015-2016 academic year. Overall, alumni, employers, current students, and site supervisors hold high perceptions of our program, as demonstrated in the following table (1 = poor; 2 = fair; 3 = good; 4 = excellent): 4 Overall Perception of the UNC School Counseling Program 3.5 3 2.5 2 1.5 1 Alumni Employers Site supervisors Students Stakeholder Groups In 2016 there were 25 graduates of the School Counseling Program. All fulfilled requirements enabling them to be recommended for school counseling licensure at the Advanced Graduate Level through the North Carolina Department of Public Instruction. Of the 25 graduates, 20 indicated as of July 2016 that they will be employed during the 2016-2017 school year as school counselors. Two graduates will assume positions as teachers and three students employment status is TBD as of July 2016. Of the 20 graduates working as school counselors, 18 are employed in North Carolina, one in Florida, and one in Georgia. Across the K-12 levels, there are nine graduates employed in high schools; three in middle schools; seven in elementary schools; and one in a K-8 school.
2016 UNC School Counseling Program Evaluation Report 3 The following sections provide an analysis of the data collected from each of the five stakeholder groups. Current Students (2016 Graduates) Exit Survey A survey requesting evaluative data about the school counseling program was sent via email in July 2016 to all 25 current students (August 2016 graduates) in the program. This questionnaire also solicited information concerning the students internship experiences, permanent contact information, employment, and elective courses. Twenty-five of the 25 students completed the survey, yielding a response rate of 100%. Respondents were asked to rate themselves on 17 skills/characteristics drawn from the CACREP standards, as they relate to their abilities as school counselors using a four-point Likert-type scale (1 = poor; 2 = fair; 3 = good; 4 = excellent). The 2016 student mean rating of the overall training received in the program was very positive (M = 3.80). On average, the highest-rated quantitative items by students were: understanding of the school counseling profession and ethical practice (M = 3.64, SD = 0.49); ability to conduct individual and group counseling (M = 3.64, SD = 0.49); and understanding of the counseling process (M = 3.64, SD = 0.49. On average, the lowest-rated quantitative items by students were: ability to conduct and interpret a variety of appropriate assessments (M = 3.12, SD = 0.74); and understanding of and ability to promote academic development (M = 3.00, SD = 0.71). Program strengths, as indicated by the students, included: the cohort model; the year-long internship in one school and integrated coursework; excellent, accessible, and invested professors with diverse backgrounds; and the relevant and applied assignments. The main areas for improvement for the program as noted by the students included: more focus on additional theories other than Solution-focused; improvement to the academic development class, including the faculty assignment; better communication regarding School of Education administrative requirements; and improved consistency and frequency of supervision at the internship site. Students were requested to give suggestions for improving the school counseling program. The following is a summary of the suggestions provided: Re-evaluate the sequencing of some courses to include Applied Investigations and Tests and Measurements in the same semester and/or moving Academic Development to the fall semester. Give students more faculty support in the job search, especially for jobs out of the Triangle region of North Carolina. Provide more practice and exposure to a variety of counseling theories Offer expanded financial aid opportunities Make Special Topics a required course.
2016 UNC School Counseling Program Evaluation Report 4 Alumni Survey A survey requesting evaluative data about the school counseling program was sent via email to all 2015 graduates of the program (N = 24). A total of 17 graduates completed the survey, yielding a response rate of 71%. Using a four point Likert scale (1 = poor; 2 = fair; 3 = good; 4 = excellent), respondents were asked to rate themselves on 17 skills/characteristics, drawn from the 2009 CACREP standards, as they relate to their abilities as school counselors. The 2015 graduate mean rating of the overall training received in the program was positive (M = 3.59, SD = 0.51). On average, the highest-rated quantitative items by graduates were: understanding of the school counseling profession and ethical practice (M = 3.76, SD = 0.44); ability to conduct group and individual counseling (M = 3.71, SD = 0.47); and understanding of the counseling process (M = 3.65; SD = 0.61) On average, the lowest-rated quantitative items by graduates were: ability to conduct research to improve the school counseling program (M = 3.14, SD = 0.53); and ability to conduct and interpret a variety of appropriate assessments (M = 2.71, SD =0.66). Graduates were also asked to provide feedback on program strengths and areas for growth. Program strengths noted by graduates included: The length (14 month) of the program, the professors, and the year-long internship The strengths-based focus of the program and full use of ASCA model Program emphasis on advocacy, leadership training, social justice, and current educational research Integration of internship and academics Emphasis on ethical guidelines and multicultural/diversity sensitivity Cohort bond, counseling skill training, and preparation Areas for growth included: More opportunities for cross level experiences More information on college and career readiness Skills are broad (vs deep) due to accelerated nature of program More exposure to different counseling theories Graduates were also requested to provide suggestions for improving the school counseling program. Suggestions included: more information on self-care and/or mandatory student participation in individual, personal counseling; more emphasis on practical techniques such as Motivational Interviewing that are effective with resistant clients; a greater focus on special education; more thorough teaching around academic systems in the school setting; and incorporating interviews with site supervisors prior to internship placement to ensure that the internship is a good fit for each student. Employer Survey To obtain feedback on the quality of the performance of our 2015 graduates and to improve the training and education of our students, alumni were sent a link to an electronic survey which they were requested to forward to their employers/supervisors. Of the 24 alumni who received the
2016 UNC School Counseling Program Evaluation Report 5 email with the link, seven of their supervisors/principals completed the survey, yielding a response rate of 30%. Using a four point Likert scale (1 = poor; 2 = fair; 3 = good; 4 = excellent), employers/supervisors were asked to rate the 2015 graduates on 17 skills/characteristics based on the 2009 CACREP standards as they relate to the graduates positions as school counselors. The employer mean rating (M = 4.0) of overall effectiveness as a school counselor reflected extremely positively on the program. On average, the highest-rated quantitative items by employers were: ability to conduct group and individual counseling (M = 4.0, SD = 0); understanding of the school counseling profession and ethical practice (M = 3.86, SD = 0.38); understanding of the nature and needs of persons of all developmental levels (M = 3.86, SD = 0.38); and understanding of the counseling process (M = 3.86, SD = 0.38). On average, the lowest-rated quantitative item by employers was: understanding of assessment, research and program evaluation (M = 3.29, SD =.49). As with program graduates, employers were also asked to provide feedback on program strengths and areas for growth. Strengths noted by employers included: the speed at which the new counselors were able to perform optimally in their new position and the consistent communication of faculty with other counselors in the school who serve as supervisors. Areas for growth included: the inclusion of information on Excel for use in organizing data, and providing students with more information on K-12 student mental health. Employers also suggested that UNC-CH provide webinars or other distance-based trainings for site supervisors who are unable to attend live trainings and orientations. Site Supervisor Survey A survey was sent via email to the site supervisors of the 25 graduate students who completed our program in 2016. This was done to obtain feedback on how the preparation of school counselors may be improved. Of the 25 surveys distributed to site supervisors, 20 were completed, yielding a response rate of 80%. Site supervisors were asked to rate the students on 17 skills/characteristics based on the CACREP standards as they related to the students position as school counselor intern using a four point Likert-type scale (1 = poor; 2 = fair; 3 = good; 4 = excellent). Overall, site supervisors perception of the program is quite high (M = 3.50) The site supervisor mean rating of their level of satisfaction with the program was high, as indicated on a rating scale of 1-10, with 10 being the highest (M = 9.15, SD = 1.63). Site supervisors were also asked to rate on a scale of 1 to 10 (1 = not likely at all; 10 = very likely) how likely they would be to recommend a colleague hire a graduate of our program; most respondents rated themselves very likely to do so (M = 9.15, SD = 1.69). On average, the highest-rated quantitative items by site supervisors were: understanding of the school counseling profession and ethical practice (M = 3.85, SD = 0.49); ability to implement a strengths-based, ASCA model school counseling program and demonstrate ethical practice (M = 3.75, SD = 0.55); and ability to conduct individual and group counseling (M = 3.70, SD = 0.57).
2016 UNC School Counseling Program Evaluation Report 6 On average, the lowest-rated quantitative items by site supervisors were: ability to conduct and interpret a variety of appropriate assessments (M = 3.10, SD = 0.72); and understanding of and the ability to promote academic development (M = 3.30, SD =.80). We also asked site supervisors to share the overall strengths of the program as well as areas for growth. Program strengths as stated by site supervisors included: the yearlong field experience which allows for a wide range of experiences; the integration of coursework and internship; good support and supervision at the university level and good communication between faculty and site supervisors; overall excellent preparedness and strength of interns as well as their cultural competency; and the focus on the ASCA National Model. Areas for growth as listed by site supervisors included: supervisors need more understanding of the expectations they should have of interns; limited knowledge of developmental needs of students in special education; time line of assignments and workload at the high school level; limited preparation in collaboration required for IEP and 504 meetings. Site supervisors were also asked to provide suggestions for improving the school counseling program. The following is a summary of the suggestions provided: Readjust and be flexible regarding due dates for some assignments to accommodate the realities of the school and level Provide students with more information on computer software used to facilitate the counseling program (e.g., Excel and Google Forms) Consider placing students at different levels each semester to allow for more cross level experience Provide more theory knowledge to help students counsel young children effectively Provide webinars for site supervisors who cannot attend orientation and provide professional development for practicing school counselors. Faculty Survey A questionnaire was sent via email to the adjunct and non-core faculty who taught in the program during 2015-2016. This was done to obtain feedback on student performance and program structure. Two of the three survey recipients responded, yielding a response rate of 66%. Faculty were asked to rate the program and students using a four point Likert-type scale (1 = poor or strongly disagree; 2 = fair or disagree; 3 = good or agree; 4 = excellent or strongly agree). The faculty mean rating of overall program effectiveness in preparing school counselors was high (M = 3.5), thus reflecting positively on the program. Faculty were also asked to rate their overall perception of the program on a scale of 1-10 (with 10 being highest); the average faculty rating was 9.0. The lowest-rated quantitative item by faculty, on average, was the program curriculum is sequenced in the best way possible (M = 3.0, SD = 0). Faculty did not note any specific strengths or areas for growth, nor did they note any suggestions for program improvement.
2016 UNC School Counseling Program Evaluation Report 7 Summary The UNC-CH School Program is highly regarded by all of the surveyed stakeholder groups. More specifically, graduates and students appreciate the cohort model; the support of the program faculty; the program s strengths-based framework; and focus on the ASCA National Model. Employers and supervisors note the high level of preparedness of the program s interns and graduates to quickly assume the role of the school counselor. The program should work to incorporate more student training on the use of relevant software used to facilitate the school counseling program. The program should provide distance-based or alternative modes of delivering training, including professional development, to site supervisors and professional counselors. Additionally, the program should consider re-sequencing some of the courses and providing students with more exposure to mental health information and additional theories and techniques, specifically for application with young children. Data from this report will be used to make quality changes and additions in areas such as curriculum sequencing, course content, site supervisor training, and professional development for school counselors.