2014 UNC School Counseling Evaluation Report 1 THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL SCHOOL OF EDUCATION SCHOOL COUNSELING, M.ED. 2014 EVALUATION REPORT Dana Griffin This report summarizes program evaluation data collected during the 2013-2014 academic year. Surveys were sent electronically to the following five stakeholders: recent graduates of the program, employers of the recent graduates, site supervisors of current students, current students, and non-core faculty who taught in our program during the 2013-14 academic year. Overall, all five groups hold high perceptions of our program, as demonstrated in the following table (1 = poor, 2 = fair, 3 = good, 4 = excellent). 4 3.5 3 2.5 2 1.5 Overall Perception of the UNC School Counseling Program 1 Alumni Employers Site Supervisors Students Faculty In 2014, there were 24 graduates of the School Counseling Program. All fulfilled requirements enabling them to be recommended for school counseling licensure at the Advanced Graduate Level through the North Carolina Department of Public Instruction. Of the 24 graduates, 20 are employed during the 2014-2015 school year as school counselors. One graduate is employed as a teaching assistant, and one graduate entered a PhD program in Cultural Studies. Two graduates did not respond. Of the 20 graduates working as school counselors, 15 are employed in North Carolina, one in Florida, one in Massachusetts, one in New Jersey, one in Colorado, and one in Illinois. Across the K-12 levels, there are 11 graduates employed as counselors in high schools, five in middle schools, three in elementary schools, and one in a K-8 school.
2014 UNC School Counseling Evaluation Report 2 The following sections provide an analysis of the data collected from each of the five stakeholders. Alumni Survey A survey requesting evaluative data about the school counseling program was sent via email to all 2013 graduates of the program (N=25). This questionnaire also solicited information concerning the graduates current employment, namely supervisor names and addresses, so that a current employer survey could be emailed to them. All 25 graduates completed the survey, yielding a response rate of 100%. Using a four-point Likert scale (1 = poor, 2 = fair, 3 = good, 4 = excellent). Respondents were asked to rate themselves on 17 skills/characteristics, drawn from the 2009 CACREP standards, as they relate to their abilities as school counselors. Each of these items also included an N/A option for not applicable. The 2013 graduate mean rating of the overall training received in the program was positive (M = 3.24, SD =.60). On average, the highest-rated quantitative items by graduates were understanding the school counseling profession (M = 3.52, SD =.51), understanding of the cultural context of counseling relationships, issues, and trends in a multicultural society (M = 3.50, SD =.59), understanding the counseling process (M = 3.48, SD =.51), the ability to conduct individual and group counseling (M = 3.48, SD =.51), and the ability to promote the personal and social development of students and build protective factors in the school environment (M = 3.46, SD =.51). On average, the lowest-rated quantitative items by graduates were understanding of assessment, research and program evaluation (M = 2.75, SD =.90); ability to conduct and interpret a variety of appropriate assessments (M = 2.76, SD =.66), ability to conduct research to improve the school counseling program (M = 2.96, SD =.68), understanding the nature and needs of persons of all developmental levels (M = 3.00, SD =.58), and understanding group counseling approaches (M = 3.05, SD =.68). Graduates were also asked to provide a list of program strengths and weaknesses. Program strengths listed by graduates included the yearlong internship placement, the cohort model, the short length of the program (14 months), the quality and enthusiasm of the professors, the strengths-based approach, hands-on experiences, and faculty supervision and availability. Areas of program weaknesses included a lack of experience/training at different levels, the varying quality of internship experiences, the lack of consistency among professors regarding supervision experiences, and lack of preparation with other theoretical approaches to counseling besides Solution-Focused counseling. Graduates were also requested to provide suggestions for improving the school counseling program. While the bulk of the comments were related to practicum/internship experiences, such as providing high quality internship placement
2014 UNC School Counseling Evaluation Report 3 sites and ensuring that professors are consistent across clinical sections, graduates also listed: Including more theoretical approaches in training and opportunities for practicing theories during practicum/internship class. Clearer expectations for graded assignments for practicum/internship assignments More flexibility in due dates for practicum/internship assignments Employer Survey To obtain feedback on the quality of the performance of our graduates who are employed as school counselors and to improve the training and education of our students, a questionnaire was sent via email to the employers of the 20 graduates of our programs who are employed as school counselors. Of the 20 surveys distributed, 8 were completed, yielding a response rate of 40%. Using a four point Likert scale (1 = poor, 2 = fair, 3 = good, 4 = excellent), employers/supervisors were asked to rate the graduates on 17 skills/characteristics based on the CACREP standards as they relate to the graduates position as school counselors. Each of these items also included an N/A option for not applicable. The employer mean rating of overall effectiveness as a school counselor reflected positively on the program (M = 3.88, SD =.35). On average, the highest-rated quantitative items by employers were understanding the school counseling profession and ethical practice (M = 3.88, SD =.35), ability to conduct individual and group counseling (M = 3.88, SD =.35), and understanding of group counseling approaches (M = 3.88, SD =.35). On average, the lowest-rated quantitative items by employers were understanding of and ability to promote career development and postsecondary access and success (M = 3.13, SD =.99), understanding of assessment, research, and program evaluation (M= 3.38, SD =.52), ability to conduct and interpret a variety of appropriate assessments (M = 3.38, SD =.52), and understanding of and the ability to promote academic achievement (M = 3.43, SD =.53). As with program graduates, employers were also asked to provide a list of program strengths and weaknesses. Graduate strengths listed by employers/employers included the following: organization skills, communication skills, professionalism, and flexibility. Graduate weaknesses listed by employers included the following: taking on too many things and becoming overwhelmed, lack of knowledge regarding testing, and cultural awareness of unique strategies for diverse populations. Employers were also requested to give suggestions for improving the school counseling program. The following is a summary of the suggestions provided by the employers/supervisors: Exposing students to accountability models Role play real world scenarios that happen in schools
2014 UNC School Counseling Evaluation Report 4 Provide instruction on cultural awareness and research-based ways to help different subgroups have success in school. Site Supervisor Survey A survey was sent via email to the site supervisors of the 24 graduate students in our program. This was done to obtain feedback on how the preparation school counselors received may be improved. Of the 24 surveys distributed to site supervisors, 22 were completed, yielding a response rate of 91.6%. Site supervisors were asked to rate the graduates on 17 skills/characteristics based on the CACREP standards as they relate to the graduates position as school counselors using a four point Likert-type scale (1 = poor, 2 = fair, 3 = good, 4 = excellent). Each of these items also included an N/A option for not applicable. The site supervisor mean rating of the quality of preparation they received to act as a site supervisor reflected positively on the program (M = 3.64, SD =.49). Further, site supervisors felt supported by program faculty (M = 3.73, SD =.55). Site supervisors were also asked to rate on a scale of 1 to 10 (1 = not likely at all, 10 = very likely), how likely they would recommend a colleague hire a graduate of our program, most respondents rated us very likely (M = 9.69, SD = 1.09). On average, the highest-rated quantitative items by site supervisors were the ability to promote the personal and social development of students and build protective factors in the school environment (M = 3.81, SD =.40), understanding of the school counseling profession and ethical practice (M = 3.77, SD =.53), ability to implement a strengthsbased, ASCA model school counseling program and demonstrate ethical practice (M = 3.77, SD =.43), and ability to counsel and advocate for diverse groups in the school context (M = 3.77, SD =.43). On average, the lowest-rated quantitative items by employers/supervisors were understanding of the nature and needs of persons of all developmental levels (M = 3.45, SD =.74), the ability to act as a leader in the school and community (M = 3.41, SD =.67), ability to collaborate and consult with other stakeholders in the school and community (M = 3.38, SD =.59), and ability to conduct and interpret a variety of appropriate assessments (M = 3.38, SD =.80). We also asked site supervisors to share the overall strengths and weaknesses of the program. Program strengths as stated by site supervisors include focus on school counseling, the yearlong internship, the use of ASCA National Model, communication between faculty and site supervisors, the two training sessions for the site supervisors, and helping students understand the role of professional school counselors. The program weaknesses as listed by site supervisors include the 14-month program, more communication between university and site supervisors, unrealistic expectations
2014 UNC School Counseling Evaluation Report 5 and timeline of what can be accomplished in real world of school counseling, and flexibility of letting students attend workshops and district meetings on class days. Site supervisors were also asked to provide suggestions for improving the school counseling program. The following is a summary of the suggestions provided: Get more input from site supervisors Teach students more than just solution-focused theory Training for site supervisors on working with weak students Need to require two years of workforce experience as a requirement of the program Faculty should ask for site supervisor input for developing practicum and internship assignments Provide more help for strengthening weak students Letting supervisors know of all student expectations Current Students Exit Survey A survey requesting evaluative data about the school counseling program was sent via email to all 24 current students in the program. This questionnaire also solicited information concerning the students site experiences, contact information, and elective courses. Out of the 24 students, all students completed the survey yielding a response rate of 100%. Respondents were asked to rate themselves on 17 skills/characteristics, drawn from the CACREP standards, as they relate to their abilities as school counselors using a four point Likert-type scale (1 = poor, 2 = fair, 3 = good, 4 = excellent). Each of these items also included an N/A option for not applicable. The 2014 student mean rating of the overall training received in the program was positive, with (M = 3.71, SD =.46). On average, the highest-rated quantitative items by students were understanding of the school counseling profession and ethical practice (M =3.58, SD =.58), ability to counsel and advocate for diverse groups in the school context (M = 3.54, SD =.51), understanding the counseling process (M = 3.54, SD =.59), ability to promote the personal and social development of students and build protective factors in the school environment (M = 3.50, SD =.51), and ability to conduct individual and group counseling (M = 3.43, SD =.51). On average, the lowest-rated quantitative items by graduates were demonstrates the ability to use evidence-based strategies and practices (M = 3.38, SD =.58), understanding the nature and needs of persons of all developmental levels (M = 3.13, SD =.68), understanding and ability to promote career development and postsecondary access and success (M = 3.08, SD =.65), understanding group counseling approaches (M = 3.08, SD =.72), and understanding assessment, research, and program evaluation (M = 3.08, SD =.58).
2014 UNC School Counseling Evaluation Report 6 Program strengths, listed by students, included the yearlong internship placement, the cohort model, the short length of the program (14 months), the strengths-based approach, the emphasis on ASCA, and dedicated and knowledgeable professors. Areas of program weakness included the length of the program (14 months), communication between faculty and between faculty and supervisors, consistency in expectations among faculty and supervisors, lack of support, more training on theories, limited exposure to other grade levels, and advocacy on behalf of the faculty for students. Students were also requested to give suggestions for improving the school counseling program. The following is a summary of the suggestions provided: More focus on theories and learning a variety of theories Better recruitment for males and minority students Better communication between professors and site supervisors More time at different levels Faculty need to serve as advocates for students Create more time for student self-care Faculty Survey A questionnaire was sent via email to the four non-core faculty who taught in the program during 2013-2104. This was done to obtain feedback on student performance and program structure. All four responded, yielding a response rate of 100%. Faculty were asked to rate the program and students using a four point Likert-type scale (1 = poor, 2 = fair, 3 = good, 4 = excellent) for 10 questions. Each of these items also included an N/A option for not applicable. The faculty mean rating of overall program effectiveness in preparing school counselors reflected very positively on the program (M = 4.00, SD = 0.00). The highest-rated quantitative item by faculty, on average, was the coursework I teach provides an important contribution (M = 4.00 SD = 0.00). The lowest-rated quantitative item by faculty, on average, was the quality of work students produce is strong and students being engaged and enthusiastic in course work (M = 3.50, SD =.58 and M = 3.50, SD =.58, respectively). Faculty suggestions for program improvement include having a stronger alignment between each semester s specific curriculum and site experiences and to help students foster a stronger sense of professional identity. Summary In summary, the program is highly regarded. Across all stakeholders, the focus on the ASCA National Model, the 14-month program, the cohort model, and utilizing a strengths-based approach are seen as strengths of the program, while a need to coordinate more with site supervisors regarding practicum and internship assignments,
2014 UNC School Counseling Evaluation Report 7 communication between site and university supervisors, and communication among university supervisors need to be improved. As far as course content, more training need to be provided on academic development, assessments, research, career planning, and developing leadership abilities. Data from this report will be used to make quality changes in areas such as program offerings, course content, site placements, supervisor training and communication, and recruitment.