COMPREHENSIVE ASSESSMENT PLAN AND REPORT

Clinical Mental Health Counseling Program COMPREHENSIVE ASSESSMENT PLAN AND REPORT 2015 Reviewed and Approved: Drs. Jungers, Kissinger, and Milburn 2/12/2016 1

Mission Statement of the CMHC Program The underlying philosophical approach in the CMH Counseling Program is consistent with Franciscan University of Steubenville s mission to promote free intellectual inquiry conducive to the development of professionals who respect and integrate human and Christian values. The program aims to: provide a strong background of experiential training in clinical mental health counseling skills required of entry-level practitioners advance students comprehension of counseling theories, models, and treatment methods develop research and testing skills needed by mental health counselors foster multicultural competence develop ethical decision-making skills promote an understanding of the physical, intellectual, social, emotional, and spiritual aspects of human development and behavior needed to competently facilitate human growth. Graduates from this program will receive the academic and experiential foundations that will enable them to specialize in a variety of counselor roles, such as community mental health counselors, pastoral counselors, drug and alcohol counselors, college counselors, and relationship counselors. (Some of these specialties would require additional post-graduate training or coursework not available through this program.) The program has the goal of providing educational foundations that enable its graduates to enter professionally satisfying careers, serve others to the best of their abilities, and develop a capacity for intellectual, professional, and personal growth. Overview of the Comprehensive Assessment Plan The assessment plan for the CMHC Program aims to help faculty regularly and systematically review student outcomes and program objectives. Assessment of student outcomes includes a review of (a) students competence in core and specialized knowledge and skills areas as established by CACREP standards and the Program; (b) students personal and professional disposition development prior to acceptance into the program, while in the program, and after graduation; and (c) student demonstration of counselor professional identity development. Assessment of program objectives includes the evaluation of program outcomes that CMHC faculty established in congruence with the mission of the Program and Franciscan University. Figure 1 shows an overview of the components of the CMHC Comprehensive Assessment Plan and their relationship to one another. 2

Figure 1: Overview of the Comprehensive Assessment Plan Mission Statement CACREP Core Areas Program Objectives CACREP CMHC SLO Standards Admissions Process Curriculum Course Level Objectives Methods Measurements Outcomes The mission statement of the Program informs the CMHC Program objectives. The objectives also are informed by the CACRE Core curriculum standards and the CACREP Student Learning Outcomes for the specialized program area of clinical mental health counseling. Both the program curriculum, which is comprised of all core courses in the CMHC Program and elective courses, and the admissions process are informed by the mission statement, program objectives, and CACREP standards. The program curriculum has been developed to enable students to demonstrate knowledge and skills competence with CACREP standards, to meet program 3

objectives, and to fulfill our program mission. Course level objectives are written to reflect the outcomes (both program objectives and accreditation standards/slos) we aim for in the CMHC Program. The methods of instructional delivery and measurements of SLOs are informed by the curriculum, accreditation standards, and common practices for instructional delivery and assessment in counseling programs. Finally, the outcomes that we measure in terms of student learning and program objectives help us to revise all aspects of our assessment plan, from specific course level objectives, to the admissions process, to program objectives and even the mission statement. Philosophy of Assessment in the Franciscan University CMHC Program The faculty is committed to a process of comprehensive program evaluation that is focused on outcomes data, collaborative in nature, and implemented at various points in a student s movement through the program. The assessment plan aims to collect and take advantage of both formative and summative data in order to assess whether or not the program is meeting its stated objectives and whether or not students are learning core knowledge and skills of professional counselors in the environments for which they are being prepared to work. The philosophy of our outcomes-based, collaborative, and ongoing assessment plan is evidenced by the following: Various points of measurement: student readiness and learning are assessed from point of entry into the program through post-graduation Multiple evaluators: students are assessed on their learning outcomes by numerous qualified stakeholders, including core and adjunct faculty in content and supervisory courses, site supervisors, and employers; the program is assessed by students, alumni, site supervisors, and internally through a university program review process Various instruments of measurement: students and the program are assessed using various tools that provide direct and indirect measures of outcomes and that are both qualitative and quantitative in nature What is Student Success in the Franciscan University CMHC Program? The successful student in the CMHC Program is one who is able to demonstrate competence in the knowledge and skills areas that are pertinent to the work of professional mental health counselors and who have evidenced the dispositions suitable to such professionals. In addition, students in the CMHC Program at Franciscan University respect Christian values and integrate a Christian understanding of the person into their clinical encounters with children, adolescents, and adults. 4

In order to concretize the concept of student success, the program has identified a number of key performance indicators that are central to measuring student success. The key performance indicators correspond with outcomes in CACREP s core and specialized standards areas. Core Area Knowledge KPI Skill KPI Professional Orientation/ TBD Ethical Practice Social and Cultural TBD Diversity Human Growth & TBD Development Career Development TBD Counseling & Helping TBD Relationships Group Counseling and TBD Group Work Assessment & Testing TBD Research & Program Evaluation TBD CMHC Specialization Area Knowledge KPI Skill KPI TBD How is Success Measured for the CMHC Program at Franciscan University? Determining the success of the CMHC Program begins with articulation of the objectives of the training program. At Franciscan University, the objectives of the CMHC Program are to help suitable and committed individuals develop the following: 1. An understanding of human nature based on a Christian philosophical perspective, thus reflecting a holistic, developmental, relational and valuing perspective on the Person, as well as a respect for self-determination and human limitation. 5

2. Professional Identity as a Clinical Mental Health Counselor, including a comprehension of Counselor roles in mental health, professional ethics and legal issues, self-care, models of treatment and supervision, and relationship to the profession, as well as understanding the impact of crisis, trauma, and substance abuse on mental health. 3. Understanding of and ability to apply: educational, advocacy, consultative, preventive and interventive strategies to fostering mental health and wellness at individual, family and larger-system levels as appropriate. 4. Integration of diversity awareness into response to client needs at individual, family and larger-system levels. 5. Understanding of and ability to apply appropriate, culturally informed individual and systemic assessment strategies to all relevant dimensions of the initial evaluation process, case conceptualization, treatment planning, and re-evaluation, including risk assessment and assessment for co-occurring disorders. 6. Understanding of and ability to apply relevant research findings and strategies to program-evaluation as well as selection, evaluation and enhancement of treatment strategies. 7. Understanding of and ability to apply current diagnostic principles, informed by comprehension of the impact of multicultural variables, co-occurring disorders, and crisis/trauma on the diagnostic process. And to offer: 8. A program of studies that covers the common core curriculum outlined by CACREP and that prepares students for licensure in Ohio and most states. 9. A concentration in Christian counseling for students desiring to learn more about applying professional skills in a Christian counseling setting. In order to operationalize success at the program level, the faculty has developed a number of key performance indicators, in addition to student learning key performance indicators, that evidence progress towards meeting program objectives. The program faculty develops outcomes goals for each year as a way to measure program success and/or indicate areas for improvement. Program Area Goal KP 1: Ensure Students Licensure Eligibility Key Performance Indicator Measure 1: 90% or greater pass rate of all students who take the National Counselor Exam (NCE) licensure exam Measure 2: Maintain CACREP accreditation in CMHC 6

KP 2: Key stakeholders confidence in the CMHC Program s ability to train students in the skills and dispositions of professional counselors KP 3: Retention and Graduation Rate specialization Measure 1: 75% or more of site supervisors and employers will rate the program at 4 or higher on the Site Supervisor and Employer Survey Measure 2: 75% of items on Site Supervisor Evaluation Form for Internship II students will be rated at a group mean of 4 Measure 1: 80% or more of students who enter the Program complete. Instruments of Assessment for Student Learning and Program Success The following are a list of the tools used to assess students enrolled in the CMHC Program and to assess overall program success. Student Learning and Disposition/Developmental Progress Assessment Tools Student Learning Outcomes Surveys (SLOS): The SLOS instruments are a set of 22 unique surveys that correspond to each of the courses in the program and that are used by the faculty to rate each student at the completion of the course. The items on the SLOS allow the professor to rate students according to each of the program and CACREP knowledge and skills competencies that are identified for the course taught. This instrument allows the program to track individual progress on student learning as well as to gather aggregate data about how well the course level SLOs are being met. BENCHMARK SCORES FOR YEAR 1 AND SUMMER COURSES: 3. BENCHMARK SCORES FOR YEAR 2 COURSES: 4. Integration Paper Survey: The Integration Paper survey is a two part survey. Part I is used to rate students on nearly 60 knowledge and skills-based SLOs and Part II is used to rate students on the SLO-based program objectives. All students are required to complete the paper at the end of their Internship II experience. The instructor of the Internship II course provides one assessment of the paper according to the survey, and another core faculty member provides a second assessment of the paper through completion of the survey. BENCHMARK SCORE FOR PART I OF INTEGRATION PAPER SURVEY: 4. Practicum Site Supervisor Evaluation: This tool is a measure of student knowledge and skills in core and specialty CACREP standards areas. All students enrolled in practicum are evaluated by their site supervisors according to this survey. The survey results 7

inform students grades in the course and also inform their advancement towards internship. Ratings below expectation are considered in faculty decisions about allowing students to advance to internship. BENCHMARK SCORE FOR PRACTICUM SITE SUPERVISOR EVAL: 2. Practicum Student Self Evaluation: This tool is a self-report measure of students perception of their own interviewing/intake skills, treatment skills, documentation skills, case management skills, agency management skills, professional identity skills, and cultural diversity skills after completion of practicum. BENCHMARK SCORE FOR PRACTICUM SELF EVAL: 2. Internship Site Supervisor Evaluation: This tool is a measure of student knowledge and skills in core and specialty CACREP standards areas. Specifically, it rates students interviewing/intake skills, treatment skills, documentation skills, case management skills, agency management skills, professional identity skills, and cultural diversity skills after completion of internship I and II. All students enrolled in internship I and II are evaluated by their site supervisors according to this survey. The survey results inform students grades in the course and also inform their advancement towards internship II (when survey is completed for Internship I students). BENCHMARK SCORE FOR INTERNSHIP SITE SUPERVISOR EVAL. FOR INTERNSHIP I STUDENTS: 3. BENCHMARK SCORE FOR INTERNSHIP SITE SUPERVISOR EVAL. FOR INTERNSHIP II STUDENTS: 4. Internship Student Self Evaluation: This tool is a self-report measure of students perception of their own interviewing/intake skills, treatment skills, documentation skills, case management skills, agency management skills, professional identity skills, and cultural diversity skills after completion of Internship I and Internship II. It differs from the Practicum Self Evaluation in that it includes additional and more nuanced items for assessment. BENCHMARK SCORE FOR INTERNSHIP I SELF EVAL: 3. BENCHMARK SCORE FOR INTERNSHIP II SELF EVAL: 4. Student Developmental Review and Dispositions Survey (SDR/DS): The SDR/DS is a 27 item tool that was developed by the department faculty to rate students development in three domains: personal/dispositional, academic, and professional. Students are rated according to this tool at three points in the program: pre-practicum, pre-internship I, and pre-internship II. The instrument provides individual data about each student s progress and also allows the program to tabulate aggregate data about how well students as a group do in their development as professionals and persons who have attitudes and dispositions appropriate to the counseling field. Items on the tool also correspond to program goals, and provide a measure of how well the program is meeting its objectives. BENCHMARK SCORES FOR SDR/DS AT PRE-PRACTICUM: 2 AND NOT MORE THAN 3 ITEMS SCORED AT 1. BENCHMARK SCORES FOR SDR/DS AT PRE-INTERNSHIP 3 AND NOT MORE THAN 2 ITEMS SCORED AT 1. BENCHMARK FOR PRE-INTERNSHIP II: 3 AND NO ITEMS SCORED AT 1. 8

Dispositions Survey (DS): The DS is a tool that is comprised on the first 18 items on the SDR/DS and is used in the admissions process to help faculty ascertain a prospective student s fit for the program. All prospective students are asked to have 3 recommenders complete the DS as part of their admissions packet. This tool adds another point in time for review of student development. BENCHMARK SCORE FOR DS: APPLICANTS WILL NOT HAVE (3) OR MORE ITEMS RATED LOWER THAN 5 BY TWO RECOMMENDERS. National Counselor Exam (NCE): The NCE is offered on campus to interested students in the semester of their graduation. Many, though not all students, opt to take the exam prior to graduation. For those that do take the exam, the program receives pass rates that inform assessment of student and program success. BENCHMARK: 90% PASS RATE FOR STUDENTS WHOSE SCORES WE RECEIVE. Program Assessment Tools Admissions Review: Core faculty review all prospective student applications, which begins the screening process to evaluating applicants potential for academic and professional success in the program. GPA, personal statements, and the DS are used to screen applicants. Graduating Student Exit Survey: The Exit Survey is administered to all graduating students in the program on a rolling basis; most students complete the survey in the spring semester. The Exit Survey is a 9 item tool that asks students to rate the program s success with delivering a program based on its mission and objectives. The survey items correspond to the program objectives and rated according to a 6 point Likert scale ranging from poor to excellent. Students are also asked to provide qualitative feedback. Alumni Survey: The alumni survey is administered every 3 years. It is used to collect demographic data such as employment and licensure information, as well as data about the program mission and objectives. Site Supervisor Survey of Program Goals/Objectives: The site supervisor survey is administered every 3 years to all of the programs recent (within 2 years) site supervisors. It is used to collect stakeholder perceptions about the program mission and objectives. The survey contains 8 items that participants rate on a Likert-type scale. Site Evaluation: The site evaluation survey is completed by all students at the end of practicum, internship I, and internship II. It provides quantitative and qualitative data about students experiences with their training sites. This data is used to provide feedback to site supervisors and to educate students about the areas of strength and weakness at training sites. 9

Student Evaluation of Clinical Supervisor: The student evaluation of clinical supervisor survey is completed by students for their university (i.e., faculty) clinical supervisors as well as for their clinical site supervisors. Students enrolled in practicum and internship rate clinical supervisors in the areas of professional skills and dispositions of clinical supervisors. IDEA (Student Evaluation of Faculty): The IDEA is the university-provided student evaluation of faculty. All faculty (core and adjunct) are evaluated by students every semester that they teach. Data from the IDEA is used as part of the faculty annual evaluation and to help inform course development and changes. Integration Paper Survey: The Integration Paper survey is a two part survey. Part I is used to rate students on (#) out of (#) knowledge and skills-based SLOs and Part II is used to rate students on the SLO-based program objectives. All students are required to complete the paper at the end of their Internship II experience. The instructor of the Internship II course provides one assessment of the paper according to the survey, and another core faculty member provides a second assessment of the paper through completion of the survey. BENCHMARK SCORE FOR PART II OF INTEGRATION PAPER SURVEY: 4. Student Developmental Review and Dispositions Survey (SDR/DS): The SDR/DS is a 27 item tool that was developed by the department faculty to rate students development in three domains: personal/dispositional, academic, and professional. Students are rated according to this tool at three points in the program: pre-practicum, pre-internship I, and pre-internship II. The instrument provides individual data about each students progress and also provides aggregate data about how well students do as a group in their development as professionals and persons who have attitudes and dispositions appropriate to the counseling field. Items # 1-5, 22, 24, 25, 26, & 27 correspond to program goals 1-7, and provide a measure of how well the program is meeting its objectives. BENCHMARK SCORES FOR PROGRAM OBJECTIVES ITEMS ON SDR/DS: Using Assessment for Student Remediation The Comprehensive Assessment Plan helps the CMHC Program faculty to evaluate the progress of students towards meeting their professional goals and towards acquiring the knowledge, skills, and dispositions that support those goals. Elements of the plan, especially the assessment tools, also support faculty in knowing when to implement their gatekeeping role, while allowing them to be transparent to students in the gatekeeping process. A full description of the remediation process for students is found in the CMHC Department Student Handbook. Below is a description of how the assessment tools are used in the remediation process. 10

Program Success Tools: These assessment tools are generally not used for student remediation. Although some of these instruments or portions of them (e.g., SDR/DS) are also used for student assessment, when they are used for program assessment, the assessment results do not impact student evaluation. Student Developmental Review and Dispositions Survey (SDR/DS): The SDR/DS is used before the start of every clinical portion of the training program as a screening tool for students readiness for practicum and internship. Students registering for practicum must have a 3.0 GPA and must receive not more than three items on the SDR/DS scored at 1. If a faculty member scores a student at 1, the faculty member must indicate the reason for the score. For three or fewer items scored at 1 a faculty member will be assigned to have a discussion with the student about the concerns noted. When greater than 3 items are scored at a 1, the student will be asked to participate in a formal review process with the department chair and/or other faculty members which includes a remediation plan. Students registering for Internship I must have a 3.0 GPA in all core courses and must not receive more than two items scored at 1. For two or fewer items scored at 1 a faculty member will be assigned to have a discussion with the student about the concerns noted. When greater than 2 items are scored at a 1, the student will be asked to participate in a formal review process with the department chair and/or other faculty members which includes a remediation plan. Students registering for Internship II must have a 3.0 GPA in all core courses and electives used towards the degree and must not have any items scored at a 1. If they receive such a score they will be required to meet with the department chair and/or faculty members to develop a remediation plan. Practicum Site Supervisor Evaluation: The benchmark score for all items on this evaluation tool is 2. When students are rated below expectation in more than 3 skill areas on this tool, they may be asked to write a narrative response to the areas in which they are rated below expectation and formulate a plan for advancing proficiency. Below expectation ratings may result in lowered grades and possible requirement to repeat the course. In all cases, final judgement of student progress will be made by the faculty supervisor and will take into account mitigating circumstances (e.g., supervisor countertransference or stubborn refusal to rate by university criteria). Internship Site Supervisor Evaluation: The benchmark score for all items on this tool for Internship I students is 3 and for Internship II students is 4. When students are rated below expectation in more than 3 skill areas on this tool, they may be asked to write a narrative response to the area in which they are rated below expectation and formulate a plan for advancing proficiency. Below expectation ratings may result in lowered grades and possible requirement to repeat the course. In all cases, final judgement of student progress will be made by the faculty supervisor and will take into account mitigating circumstances (e.g., supervisor countertransference or stubborn refusal to rate by university criteria). 11

Timelines for Assessment The assessment timelines guide the systemic and regular cycle of student and program evaluation. Table 1 shows the timeline for gathering and reviewing data related to the systematic developmental assessment of student progress. Table 2 shows the timeline for gathering and reviewing data related to student learning outcomes and key performance indicators for student learning. Table 3 shows the timeline for gathering and reviewing data related to program evaluation. Faculty appreciate and understand that these various assessment processes inform one another, as is indicated in Figure 1: Overview of the Comprehensive Assessment Plan. 12

Table 1: CMHC Program Evaluation Plan & Calendar for Systematic Developmental Assessment of Student Progress Revised and approved by CMHC Faculty: 11/6/2015 Assessment Periods Pre-Acceptance: purpose is to determine candidate s fit for the program Data Collected 1. Application Packet: -Evidence of 2.5 GPA (with acceptable GRE/MAT) or 3.0 GPA -Evidence of 3.3 GPA for those entering through the accelerated track -Personal statement -Disposition survey completed by 3 recommenders -Resume -Transcript -Evidence of appropriate undergrad degree or pre-reqs Data Collected by Whom 1. Grad Admissions 2. Assigned faculty Evaluators of Data 1. Faculty Review committee (all core faculty) 2. Qualitative Interviews or Survey: compiled by Dr. Kissinger & reviewed by all faculty Period of Data Collection 1. Ongoing (with each new application) 2. Every 2 years: 12/15/2015 2. Survey of students entering through accelerated track Pre-Practicum: purpose is to assess student readiness to enter practicum 1. Student Developmental Review and Disposition Survey collected for every student enrolled to take practicum to assess personal, academic, and professional readiness 2. Grades (minimum 3.0 avg.) 1. All core faculty who taught practicumenrolled students in the semester prior to practicum 2. Core faculty 1. Core faculty 2. Core faculty 1. Semester prior to students taking prac. 2. End of semester prior to prac. 13

Pre-Internship I: purpose is to assess student s readiness for internship expectations 1. Student Developmental Review and Disposition Survey collected for every student enrolled to take internship I 2. Site supervisor evaluation of practicum performance 3. All practicum course assignments, including recordings and papers and participation in supervision 4. Practicum student learning outcomes ratings 1. All core faculty who taught internship I- registered students in the semester prior to Internship I 2. Site supervisors 3. Faculty supervisor 4. Faculty supervisor 1. Core faculty 2. Core faculty 3. Core faculty 4. Core faculty 1. Semester prior to students taking Internship I 2. End of practicum 3. Throughout practicum 4. End of prac and before start of internship Pre-Internship II: purpose is to assess student s readiness for internship expectations Post-Internship 1. Student Developmental Review and Disposition Survey collected for every student enrolled to take Internship II 2. Site supervisor evaluation of Internship I performance 3. All Internship I course assignments, including recordings and papers and participation in supervision 4. Internship I student learning outcomes ratings 1. Site supervisor Evaluation 2. All Internship II 1. All core faculty who taught internship II-registered students the semester prior to Internship II 2. Site supervisors 3. Faculty supervisor 4. Faculty supervisor 1. Site supervisors 1. Core faculty 2. Core faculty 3. Core faculty 4. Core faculty 1. Core faculty 2. Faculty 1. Semester prior to students taking Internship II 2. End of Internship I 3. Throughout Internship I 4. End of Internship I and before start of internship II 1. End of Internship II 2. By end of Internship II 14

assignments, including recordings, and participation in supervision 3. Integration Paper Survey 4. Internship II Student Learning outcomes ratings 2. Faculty supervisor 3. Faculty supervisor 4. Faculty supervisor supervisor 3. Faculty supervisor and 1 external faculty reader 4. Core faculty 3. By end of Internship II 4. End of Internship II Table 2: Timeline for Evaluation of Student Learning Outcomes Assessment Periods Year 1 Courses and Summer content courses Data Collected Student Learning Outcomes Surveys for: CSL 502, 504, 607, 621, 505, 520, 608, 624, 630 PC, 630 PE, 630 PB, 630 PK, 636 Data Collected by Whom Faculty instructor Evaluators of Data Faculty Review committee (all core faculty) Period of Data Collection End of semester Year 2 Courses Student Learning Outcomes Surveys for: CSL 503, 623, 630 PJ, 521, 630 PL, 630 PD, 630 PA, 501 Faculty instructor Faculty Review committee (all core faculty) End of semester Internship I & II Student Learning Outcomes Surveys for: CSL 625 & 626 Faculty instructor Faculty Review committee (all core faculty) End of semester 15

Table 3: Program Assessment Timeline Assessment Tool Admissions review SDR/DS Prac. Self Evaluation Internship Self Evaluation Integration Paper Survey Graduating Student Exit Survey Site Supervisor Survey Alumni Survey Student Eval. of Clinical Supervision Pre- Orientation Core Faculty & Admin. Assistant Every 3 years (next: 2018) Every 3 years (next: 2016) Practicum Orientation Pre- Practicum Core faculty Practicum (Final Faculty Supervisor Administers Prac/Internship Coordinator Administers Pre- Internship Core Faculty Internship I (final) Faculty Supervisor Administers Prac/Internship Coordinator Administers Pre- Internship II Core Faculty Internship II (final) Faculty Supervisor Administers Faculty supervisor plus one additional core faculty Prac/ Internship Coordinator Administers Last Semester Accessible on Bb every semester 16

Program Outcomes Goals for 2015 1. Students in the CMHC Program will demonstrate a competent grasp of the knowledge deemed central to professional counselors by the Program and by CACREP standards. a. Measure 1.1: At least 75% of learning outcomes will be rated 4 or above on the knowledge standards as measured by the Integration Paper Survey. b. Measure 1.2: At least 75% of learning outcomes will be rated 3 or better on knowledge standards for first year and summer courses. c. Measure 1.3: At least 75% of learning outcomes will be rated 4 or better on knowledge standards for 2 nd year courses 2. Students in the CMHC Program will demonstrate basic counseling skills by the completion of CSL 624 Practicum in Mental Health Counseling. a. Measure 2.1: At least 75% of students in practicum will receive a rating of 2 or better on all items on the Practicum Site Supervisor Evaluation 3. Students in the CMHC Program will demonstrate counseling skills commensurate with entry-level professionals by the completion of Internship II. a. Measure 3.1: At least 75% of items rated in Internship I will receive a rating of 3 or better on the Internship I Site Supervisor Evaluation b. Measure 3.2: At least 75% of items rated in Internship II will receive a rating of 4 or better on the Internship II Site Supervisor Evaluation c. Measure 3.3: At least 75% of skills items rated in on the SLO survey for Internship II will be at a rating of 4 or better. 4. Students in the CMHC Program will demonstrate confidence in their ability to practice as a professional counselor at an entry level. a. Measure 4.1: The mean rating on all items on the Exit Survey will be 4 (above average) or better. b. Measure 4.2: The mean rating of all items on the Internship II Self Evaluation will be 4 or greater. 5. Stakeholders in the Program will express confidence in the Program s ability to train students into the professional skills and dispositions of mental health counselors. a. Measure 5.1: At least 75% of items rated by site supervisors in the Site Supervisor Survey of Program Mission and Goals in 2015 will rate the program at 4 or better. 17

Data Related to Program Outcomes Goals for 2015 See Attached Summary of Assessment Data and Response to 2015 Goals Goal Measure Outcome 1 Competent grasp of core and specialty area knowledge 1.1 Integration Paper Survey Data not used for this year (testing instrument) 1.2 Knowledge SLOs on 1 st year courses MET 81.7% of items rated at 3 or better 1.3 Knowledge SLOs on 2 nd year courses MET 77% of items rated 4 or better 2 Demonstration of basic counseling skills by end of Practicum 3 Demonstration of entry-level counseling skills 2.1 Rating on Practicum Site Supv. Eval. MET 100% of items rated at 2 or better 3.1 Rating on Internship I Site Supv. Eval. MET 100% of items rated at 3 or better 3.2 Rating on Internship II Site Supv. Eval. MET 100% of items rated at 4 or better 3.3 Rating on SLO skills items for Internship II UNMET > 75% of items rated at 4 or better 4 Demonstrate confidence in entry-level skills 4.1 Exit survey data was not captured for this year 5 Stakeholders expressed confidence in program s training of students into appropriate skills and dispositions 4.2 Benchmark mean 4 rating for all items 5.1 Site supervisor ratings on site supervisors survey of mission/goals MET Mean rating of all items was 4.4196 MET 100% of items rated at 4 or higher 18

Reflections and Action Steps Related to Student Learning and Program Outcomes Goals Goal 1: Core and Specialty Knowledge Assessment Data Summary SLOs for 1 st Year Courses: For the 2014-15 year 81.7% of knowledge items were rated at the benchmark score of 3 or better. Assessment Data Summary SLOs for 2 nd year courses: For the 2014-15 year 77% of knowledge items were rated at the benchmark score of 4 or better. This outcome includes the outlier of CSL 521, which was rated before the benchmark scores were set and for which all of the SLO ratings were below the benchmark (0%). Not factoring CSL 521 into the average number of SLO met results in 85% of ratings being met at the benchmark 4 level. Action: Low rated learning outcomes that deserve attention include: a) knowledge related to addictive behaviors and effects upon individual/family development (CSL 505) and knowledge of addiction through the disease concept (630PC); b) knowledge of EBP that can be used in case conceptualization with families and couples; knowledge related to diagnosis process during crisis response (CSL 630 PC) and of crisis intervention models (CSL 625). Faculty can brainstorm how to more intentionally bring this information into the core courses. Goal 2: Basic Counseling Skills Assessment Data Summary for Practicum Site Supervisor Eval.: For the 2014-15 and 2015-16 years (for available data), benchmark ratings were exceeded on the measures practicum site supervisor ratings. Students appear to be demonstrating skills commensurate with their training levels in practicum. Action: No particular action is warranted based on ratings; however, practicum supervision might be another opportunity for integration of modules on clinical skill development in low-rated entry-level skills areas. Goal 3: Entry- Level Counseling Skills Assessment Data Summary for Internship I & II Site Supervisor Eval: For the 2014-15 and 2015-16 years, benchmark ratings were exceeded on both measures internship I and II site supervisor ratings. Students appear to be demonstrating skills commensurate with their training levels in internship I and internship II. What stands out, however, are the scores in the area 19

of crisis/trauma assessment and intervention. Not all students appear to be rated in those areas, which suggests that supervisors did not have enough data on which to evaluate them. Perhaps we need to find other opportunities for how to incorporate this information into their training. In addition, review of SLOs skills areas in CSL 625 and 626 provide suggestions for action steps based on items rated below benchmark. Action: Integrate more opportunities for students to practice crisis response skills in internship supervision courses and tailor site supervisor clinical supervision training to include information about how to supervise through crisis events and how to involve students, to the extent possible, in crisis response on site. Integrate more opportunities for students to screen for addiction and apply appropriate counseling techniques with addiction issues; and demonstrate skills in suicide risk assessment. Goal 4: Student Confidence in Entry-Level Skills Assessment Data Summary on Internship II Self Eval: The mean rating of all items evaluated on the Internship II Self Evaluation for 2015 was 4.4196. Students appear confident in their ability to enact core skills identified with beginning counselor practices (as determined by CACREP CMHC specialty standards and program standards). Goal 4 was met as measured by this instrument. In addition, students self- confidence after completing internship II seems to have trended up over the past 8 years. Action: No action is warranted, though the Program might consider using a single measure to evaluate self-efficacy and confidence across the duration of the program. Goal 5: Key Stakeholder Confidence in Program s Training Goals and Mission Assessment Data Summary on Site Supervisor Survey of Program Mission and Goals: 100% of items on the site supervisor survey of program goals and mission were rated at a 4 or higher on a 5 point Likert scale (4=agree; 5= strongly agree). Action: None at this time. 20

Program Goals KPIs KPI 1: NCE Pass Rate MET - 94.5% pass rate for students in 2014-2015 KPI 2: Stakeholder confidence MET 100% of items on site supervisor survey of program goals and mission statement were rated as relevant KPI 3: Retention (80% or greater completion rate) MET in 2014-2015 90.5% of students who entered the program completed 21