QUALITY ENHANCEMENT PLAN 2017 ANNUAL REPORT NORTHWESTERN STATE UNIVERISTY OF LOUISIANA

Transcription

1 QUALITY ENHANCEMENT PLAN 2017 ANNUAL REPORT NORTHWESTERN STATE UNIVERISTY OF LOUISIANA

2 Program Learning for Life: Experience Your Future (QEP) Prepared by: William Brent & Chris Gilson Date: July 7, 2017 Approved by: Vickie Gentry Date: July 24, 2017 Northwestern Mission. Northwestern State University is a responsive, student-oriented institution that is committed to the creation, dissemination, and acquisition of knowledge through teaching, research, and service. The University maintains as its highest priority excellence in teaching in graduate and undergraduate programs. Northwestern State University prepares its students to become productive members of society and promotes economic development and improvements in the quality of life of the citizens in its region. Learning for Life (QEP) Mission. The mission of the Learning for Life QEP is to prepare students to transfer theory into practice as they transition from University settings to a career or advanced study in graduate or professional schools. Student Learning Outcomes: SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. Measure 1.1 During the QEP Pilot, each Pilot Program assessed its experiential learning activities using an electronic rubric aligned to each Learning for Life SLO. Methodology Pilot Program QEP Coordinators used an electronic rubric aligned to each Learning for Life SLO. The electronic rubric was developed in support of the standardized rubric found in Section X of the Learning for Life QEP document. The electronic rubric included items concerning whether students accomplished each of the following for SLO 1: connect concepts through an interdisciplinary perspective ; adapt and implement previously learned knowledge and skills to new contexts, situations, or scenarios ; communicate effectively ; expand sense of future self through reflection on participation in experiential learning processes ; demonstrate professional characteristics and behaviors. Each of these items included benchmarks for Developing, Basic, Mastery, and Advanced levels. Target

3 The target is for at least 50% of students to perform at benchmark Level 3 or Level 4 on 50% of the items contained in the rubric for SLO 1. Findings Target Met. Analysis Program Program Finding Learning Experience (SLO 1) Early Childhood Education Target Met Internship Elementary Education Target Met Internship Health and Exercise Science Target Met Internship Hospitality Management & Tourism Target Met Internship Music Target Met Performance-Based Project Radiologic Science Target Met Internship Scholars College Target Met Research All Pilot Programs successfully met the target established in the Learning for Life QEP. This is expected since the Pilot Programs all have a long history of experiential learning activities and capstone coursework. It is unclear whether programs are utilizing the rubric for SLO 1 in a similar fashion. Decisions Training will be provided to standardize the implementation of QEP rubrics and assessment tools. Additional refinement of rubric language and terminology may be needed to address curricular differences in research and performance-based programs. The target set for each SLO may be too low for established QEP programs. Beginning with the annual assessment, higher targets will be identified for programs that have completed one year of successful QEP implementation. Pilot Program QEP Coordinators have identified QEP-related actions for consideration by program faculty as defined in each program assessment. SLO 2. During the capstone experiential course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. Measure 2.1 During the QEP Pilot, each Pilot Program administered a student survey to assess whether students were reflecting critically to link theory with practice and to develop new applications of knowledge. Methodology

4 Pilot Program QEP Coordinators used an electronic rubric aligned to each Learning for Life SLO. The electronic rubric was developed in support of the standardized rubric found in Section X of the Learning for Life QEP document. The electronic rubric included items concerning whether students accomplished each of the following for SLO 2: communicate effectively using appropriate conventions of language and correct format(s) ; connect prior learning to changes that are a direct result of the experiential learning process ; revisit prior learning to apply knowledge and skills in new and innovative ways ; assess what they have learning about themselves as members of a broader community ; assess what they have learned about themselves as individuals. Each of these items included benchmarks for Developing, Basic, Mastery, and Advanced levels. Target The target is for at least 50% of programs to perform at benchmark Level 3 or Level 4 on 50% of the items contained in the rubric for SLO 2. Findings Target Met. Analysis Program Program Finding Learning Experience (SLO 2) Early Childhood Education Target Met Internship Elementary Education Target Met Internship Health and Exercise Science Target Met Internship Hospitality Management & Tourism Target Met Internship Music Target Not Met Performance-Based Project Radiologic Science Target Not Met Internship Scholars College Target Met Research Five of the seven Pilot Programs successfully met the target established in the Learning for Life QEP. This is expected since the Pilot Programs all have a long history of experiential learning activities and capstone coursework. It is unclear whether programs are utilizing the rubric for SLO 2 in a similar fashion. Two programs did not meet the target established in the QEP: Music and Radiologic Science. In Music, this was the result of some confusion concerning the rollout of the QEP and its updated rubrics. In Radiologic Science, this was the result of an implementation process that did not include all appropriate coursework. Decisions

5 Training will be provided to standardize the implementation of QEP rubrics and assessment tools. Additional refinement of rubric language and terminology may be needed to address curricular differences in research and performance-based programs. The target set for each SLO may be too low for established QEP programs. Beginning with the annual assessment, higher targets will be identified for programs that have completed one year of successful QEP implementation. Pilot Program QEP Coordinators have identified QEP-related actions for consideration by program faculty as defined in each program assessment. For example, the Department of Music will discuss and implement ways to add a critical reflection post-recital experience to standardize the process of assessing SLO 2. This may provide a model for other programs to follow as they implement the QEP. The School of Allied Health will add an additional course, RADS 4611 (Clinic V), to the capstone experience to capture additional experiential learning and student reflection exercises. To encourage higher response rates on survey instruments, the QEP Director and the UAC will utilize survey tools that track student, faculty, and administrator participation. Measure 2.2 During the QEP Pilot, each Pilot Program administered a student survey to assess whether students were reflecting critically to link theory with practice and to develop new applications of knowledge. Methodology The Student Reflection Survey included items concerning each of the following: prior knowledge, transferable skills, improvement of skills, development of new skills, skills needing improvement, professional skills (prior to and following capstone experience), professional communication, professional strengths and weaknesses, and personal and community impact. The survey items include open-ended response, multiple-selection, and Likert-type questions. Two attached tables provide evidence of this process, demonstrate improvement in professional skills over the course of the capstone experience, and help identify differences between the three piloted experiential practices (2017 QEP Assessment Student Reflection Survey Summary and 2017 QEP Assessment Student Reflection Survey Likert Summary) Target Gather baseline data concerning student self-reflection from all seven programs participating in the QEP Pilot. Findings Target met. Analysis

6 As identified in the attached Student Reflection Survey Summary and Student Reflection Survey (Likert) Summary, the seven pilot programs gathered baseline data that will assist future programs in determining critical concepts for emphasis and potential challenges. For example, in all three experiences (Undergraduate Research, Internship, and Performance-Based Projects), students identified Time management as the professional skill still needing the most improvement after the capstone experience. The pilot Performance-Based program, Music, recorded the least amount of change in the attainment of professional skills over the course of the experience. This is likely a consequence of differing interpretations of the questions which may be addressed through training. Decisions The QEP Program will continue gathering data using these instruments, but it will work with QEP Program Coordinators to encourage higher response rates and clearer instructions. Data will be shared with Pilot Program Coordinators and all Program Coordinators preparing for the QEP implementation process. To encourage higher response rates on survey instruments, the QEP Director and the UAC will utilize survey tools that track student, faculty, and administrator participation. Service Outcomes: SO 1. The QEP Director, Task Force, and Teams will provide guidance to programs implementing and assessing experiential learning activities. Measure 1.1 During the QEP Pilot, the QEP Director administered a Faculty & Administrator Survey to assess whether program outcomes matched expectations. Methodology The Faculty & Administrator Survey included items concerning each of the following: growth; preparation; authenticity; reflection; real-time experience; environment; assessment; evaluation; reporting; and recommendations for QEP program improvement. The survey items include open-ended response and Likert-type questions. Target Gather baseline data concerning faculty and administrator perceptions of the Learning for Life QEP. Findings Target Met.

7 Analysis There was a very limited response to the faculty and administrator survey (n=5). A majority of the respondents selected Agree or Strongly Agree to all of the statements included in the Likert-type item: I believe the intended experiences allowed students to grow in knowledge and to demonstrate that knowledge and/or new skills (80%); I believe the students participating in the program were adequately prepared for the experiential learning experience (80%); I believe the experience was authentic and allowed participants to respond to real situations (80%); I believe participants were able to reflect during the course of the experience and allowed to adjust/make changes based on the situations faced (80%); I believe students and facilitators were wellprepared with background information about each other and the environment in which the experience would take place (80%); I believe the experience provided a "real time experience and that adequate feedback related to the learning intentions was provided during the process (80%); I believe adequate assessment and evaluation tools were utilized ruling the experience to allow for reflection on the specific learning goals and objectives (55%); I believe adequate reporting of shared impressions occurred during the experience (70%). Based on this data, it is evident that additional training will be necessary to ensure that assessment and evaluation tools and processes are adopted and administered in a uniform and timely manner. Decisions The QEP Program will continue gathering data using this instrument, but the QEP Director will encourage higher response rates. Data will be shared with University administrators, Pilot Program Coordinators, and all Program Coordinators preparing for the QEP implementation process. The QEP Director, with the help of the newly-hired University Assessment Coordinator (UAC), will place special emphasis on addressing faculty and administrator concerns regarding assessment and evaluation processes. To encourage higher response rates on survey instruments, the QEP Director and the UAC will utilize survey tools that track student, faculty, and administrator participation. Comprehensive Summary of Key Findings and/or Decisions The purpose of the QEP Pilot was to begin the process of implementing, aligning, and assessing QEP student learning outcomes in a controlled setting seven programs that already practiced experiential education. Included in the Plan were two student learning outcomes (SLOs) associated with professional practice and reflection and one service outcome concerning the overall QEP process. The Pilot Programs met all three targets associated with the SLOs; the QEP met the target associated with its service outcome. The QEP Director and Assistant Director have identified several key findings. First, the targets for each SLO may be too low for established QEP programs, or for programs that have been utilizing experiential learning practices for several years. Beginning in

8 , the QEP Director and the University Assessment Coordinator (UAC) will identify higher targets for programs that have completed one year of successful QEP implementation. It is too early in the process to consider raising targets for all forthcoming QEP program applicants. A second key finding concerns the need for additional training to standardize the implementation of QEP rubrics and assessment instruments. Several programs remarked in their assessment reports that they were uncertain about the appropriate use of QEP rubrics as they related to their specific disciplines. To address this need, the University will be hosting a QEP retreat during on-call week at the beginning of the Fall 2017 semester. Faculty representing all departments will participate in professional development concerning experiential education, capstone best practices, rubrics, and assessment. Additional refinement of rubric and instrument language and terminology may be needed to address curricular differences in programs that emphasize research or performance-based projects and experiences. Several programs have proposed innovative methods of addressing the QEP SLOs that may serve as models for future implementation. For example, the Department of Music will discuss and implement ways to add a critical reflection post-recital experience to standardize the process of assessing SLO 2. A third key finding concerns the low response rates on QEP survey instruments. To address this shortcoming, the QEP Director and the UAC will utilize survey software that can track student, faculty, and administrator participation. The QEP Director and Implementation Team will also encourage programs to incorporate the QEP Survey Instruments as mandatory elements of their capstone courses. Data gathered through these methods will be shared with University administrators, Pilot Program Coordinators, and all Program Coordinators preparing for the QEP implementation process to promote continuous improvement of the QEP process.

9 2017 QEP Assessment Student Reflection Survey Summary

10 2017 QEP Pilot Program, Student Reflection Survey Summary Multiple-Selection Items Internship Programs (n=35) Research Programs (n=20) Performance-Based Programs (n=14) Q2 What transferable skills did you bring to the [capstone experience]?* Q3 What skills did you improve during the [capstone experience]?* Q4 What new skills did you develop during the [capstone experience]?* Q5 What skills do you still feel you need to improve after the [capstone experience]?* Q6 In manners of professional communication, describe the ways in which you have communicated effectively during your [capstone experience]?* Q19 Select three words that best describe your reflections about your experiences with the [capstone experience]?* * Top 3 responses included for each. Listening (86%) Time management (83%) Oral communication (78%) Oral communication (83%) Problem solving (75%) Time management (69%) Plan/organize events (41%) Presentation (38%) Clinical & laboratory (38%) Time management (32%) Oral communication (32%) Creative & critical thinking (32%) Oral (97%) (74%) Telephone (60%) Determined (79%) Excited (73%) Prepared (67%) Online & library research (85%) Written communication (80%) Work independently (80%) Creative & critical thinking (80%) Find and read technical or scholarly sources (80%) Time management (85%) Written communication (70%) Presentation (65%) Develop project/research proposal (70%) Formulate thesis questions (40%) Presentation (35%) Follow professional style guidelines (35%) Time management (73%) Analyze data (26%) Problem solving (21%) Listening (21%) Presentation (21%) Oral (100%) (95%) Written (80%) Determined (75%) Overwhelmed (70%) Excited (55%) Time management (85%) Listening (85%) Collaboration (85%) Time management (92%) Listening (85%) Problem solving (71%) Plan/organize events (71%) Creative & critical thinking (71%) Time management (64%) Presentation (42%) Follow professional style guidelines (35%) Time management (64%) Plan/organize events (57%) Delegate/instruct others (42%) Collaboration (42%) Oral communication (42%) Oral (92%) (78%) Written (35%) Telephone (35%) Excited (100%) Prepared (77%) Determined (77%)

11 2017 QEP Assessment Student Reflection Survey Likert Summary

12 2017 QEP Pilot Program, Student Reflection Survey Summary Internship Programs Research Programs Performance-Based Programs Likert-Type Items (n=35) (n=20) (n=14) Pre (Q7)* Post (Q8)* Diff. % Δ Pre (Q7)* Post (Q8)* Diff. % Δ Pre (Q7)* Post (Q8)* Diff. % Δ Time management % % % Ability to delegate/instruct others % % % Ability to plan/organize events % % % Problem solving % % % Collaboration % % % Analyze data/audit % % % Manage money/budgets % % % Oral communication % % % Written communication % % % Listening % % % Presentation % % % Ability to give/receive feedback % % % *Weighted Average, Scale 1-3 (Needs Improvement, Average, Excellent) Q7 "Rate your professional communication skills as you began the [capstone] event as either 'Needs Improvement', 'Average', 'Excellent.'" Q8 "Rate your professional communication skills as you complete your [capstone] event."

13 2017 QEP Pilot Program Assessment Reports

14 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Pilot Program. Liberal Arts BA Research Assessment Louisiana Scholars' College Prepared by: Margaret E. Cochran Date: June 6, 2017 Approved by: Date: Student Learning Outcomes (SLO). SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. (Aligned with Principle 1-Intention and Principle 2-Preparedness and Planning) SLO 2. During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. (Aligned with Principle 4-Reflection and Principle 6-Monitoring and Continuous Improvement) Measure. A six-hour senior thesis is a graduation requirement for the college's Bachelor of Arts with a major in Liberal Arts and a concentration in Scientific Inquiry (820A), Humanities and Social Thought (820B), Fine and Performing Arts (820 C), Foreign Languages (820D), and Philosophy, Politics, and Law (820F). Students in any of the Scholars' College joint majors, offered in collaboration with other programs in the University, must also complete a thesis. The thesis experience is divided into three courses: SBUS/SLSC Thesis Research Methods (1 credit hour in spring of the junior year), SBUS/SFPA/SHUM/SSCI 480T--Thesis Research (2 credit hours in the fall of the senior year), and SBUS/SFPA/SHUM/SSCI 482T--Thesis (3 credit hours in the spring of the senior year). Students produce a proposal in the first course and receive a letter grade at that time. The second course typically results in deferred grading (IP), which is corrected when the archival copy of the thesis is turned in. Methodology. In Spring 2017, 20 students completed their thesis projects. Students were assessed twice: at the oral defense and after the archival copy was submitted. At the oral defense, the first and second reader and the Director of the Scholars' College (or designee) independently rated the student according to the rubrics for SLO1 and SLO2 (Learning for Life (LfL), pp ), based solely on the performance and information given during the defense. First readers were instructed to ask leading questions addressing rubric benchmarks if they were not apparent from the student's presentation and the information did not arise independently in the question and answer period. Once the archival copy was completed, the first and second readers were asked to independently rate the student on each rubric benchmark, based upon the defense, the archival document, and their own interactions with the student through the thesis process.

15 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 The responses for each benchmark were averaged for each student. Individual ratings that were omitted were not included in the averages. Target. The goal of the program is for at least 50% of students to perform at level three (3) Mastery or level four (4) Advanced on each rubric item for each SLO. The overall benchmark is for 50% of students to have an average score of 3 or greater (LfL, p. 41). Finding. Target Met. Scholars' College students exceeded this goal (Table 1). The worst performance was on SLO 2 (During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflections), where 4 students (20%) averaged lower than 3.0 on Benchmark 1 (Students will communicate effectively using appropriate conventions of language and correct format(s)). The number of students who averaged below a 3.0 on each of the other rubric items ranged from 1 to 3. Table 1. Number of Pilot Program Undergraduate Research Participants Scoring below 3.0 Note: N = 20 SLO1 SLO2 benchmark # freq benchmark # freq % % % % % % % % % % 6 1 5% Students were also evaluated on their average scores overall rubric items. The average of this score was 3.58 (SD = 0.478, N = 20). Again, the benchmark was exceeded; 85% of the population had a mean score above 3.0. In fact, the 95% confidence intervals for the mean score for each of these 17 students was also above 3.0. Analysis. The pilot study revealed a number of issues with the application of the rubric. First, while the rubric was appropriate for courses in which a reflection component was administered, this was not the case for the final Scholars' College thesis course. If we want to continue to administer this form, the Scholars' College needs to incorporate a reflection piece in the capstone sequence. In particular, the rubric seemed more

16 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 appropriate for NSU's typical internship or a project involving clients or participants. (And not all internships involve clients.) Thus, this may become an issue in the future, even for departments choosing internships. Scholars' College faculty (and second readers from other departments) received no training on interpreting the rubric. As a result, 90 responses (of 1089) were omitted. Second readers from outside the College did not rate any responses for one or both Student Learning Outcomes as well as for an occasional single benchmark for seven (7) students (a total of 59 items (5.4%)). Most indicated that they did not feel qualified to rate the students, even though they were asked to rate only their own interactions with the students, the thesis defense, and the final written document. A single Scholars' faculty member was responsible for 39 omitted responses (3.6%) because she felt the benchmarks were "irrelevant." This included SLO2 benchmarks 4 (Students will assess what they learned about themselves as members of a broader community) and 5 (Students will assess what they have learned about themselves as individuals). Since both of these points are usually covered in the thesis defense question period (with every student able to identify with the community of researchers in their fields), this clearly should not be an issue. It would be beneficial to "translate" the rubric to point out skills and dispositions that occur in the research setting. This would make it much easier to rate students. The missing responses were distributed across individuals. Due to the redundancy in the procedure, all benchmarks had at least one rating and that only occurred once. Most ratings were relatively consistent within students. The average width of the 95% confidence interval for overall rating was Although this cohort far exceeded the target, it is too early in the process to change the goal, based on this group alone. Before making such a change we must ensure that the faculty evaluators are using the rubric consistently and as intended. In addition, the student population is not a random sample of graduating seniors and, as honors students, may be expected to perform at a higher level. This population has significantly higher ACT scores and high school GPAs than the student body as a whole, and must have a cumulative GPA of 3.0 or higher to graduate. Weaker students often self-select and leave the College prior to commencing on the thesis project. Decision / Recommendations. Scholars' College needs to incorporate a reflection piece in the capstone sequence. To demonstrate growth through the process, it is recommended that reflection pieces be collected, at a minimum, at the beginning and end of the Research Methods course, during the Thesis Research course, and at the end of the Thesis course in the final semester. We also need to ensure we train faculty evaluators on interpreting the rubric for consistent application. It is recommended the rubric be translated to point out skills and dispositions that occur in the research setting.

17 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Pilot Program: Bachelor of Science in Radiologic Sciences Prepared by: Joel Hicks Date: June 28, 2017 Approved by: Date: Student Learning Outcomes (SLO). SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. (Aligned with Principle 1-Intention and Principle 2-Preparedness and Planning) SLO 2. During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. (Aligned with Principle 4-Reflection and Principle 6-Monitoring and Continuous Improvement) Measure. The final two clinical courses in the BSRS program (RADS 4611/RADS 4711) serve as an internship experience for students. Throughout the program, students are introduced to a variety of advanced imaging modalities. During these final two semesters, students choose the imaging modality that most interests them. Throughout the internship experience, students are evaluated by both radiologic technologists working in the modality and by NSU faculty. Students are evaluated on their abilities to integrate previous knowledge into the new experience, their abilities to communicate with patients and staff, and their abilities to critically think. (As this pilot program was begun in the spring 2017 semester, students were only evaluated in one semester, instead of the final two semesters) Methodology. Using the evaluation rubrics provided by the QEP committee, students were evaluated twice in the spring 2017 semester (midterm and final). NSU faculty used evaluations completed by both technologists in the various modalities and their own observations of students to address each of the benchmarks on the QEP rubrics. Target. The goal of the program is for 100%% of students to perform at level three (3) Mastery or level four (4) Advanced on each rubric item for each SLO. The overall benchmark is for 50% of students to have an average score of 3 or greater (LfL, p. 41).

18 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Analysis. The QEP for the BSRS program will consist of two clinical internship courses: RADS 4611 and RADS Because of when the QEP projects were implemented, however, only the RADS 4711 course was used in this pilot project. Regarding the data, three of the six items in SLO 1 were unmet during the midterm evaluations. Four of the five items in SLO 2 were unmet. It was felt by the faculty, however, that when the RADS 4611 course was added to the QEP, the number of students not meeting the benchmark would improve. Also, the evaluation tools used to complete the QEP rubrics were not completely similar. Therefore, faculty were required to make educated guesses as to some of the items and the students mastery of each item. The improvement in outcomes for both SLO 1 and SLO 2 during the final evaluations seemed to confirm the faculty s belief that once students (and faculty) were more fully aware of the expectations, all would improve. Decision / Recommendations. It was difficult to make recommendations based on this data set. In general, faculty were able to remind students of the importance of many of the items on the rubric (communication, reflection on past studies, innovative thinking, etc.) Based on the necessity to find evaluation tools within the class that more closely match the QEP rubrics, the QEP coordinator has created a better product that will be incorporated in both RADS 4611 and RADS It is believed that these tools and courses will benefit students in their pursuit of imaging science jobs, which will match the intent of the QEP for Northwestern State University, Learning for Life.

19 1 BSRS Learning for Life Midterm Evaluations Northwestern State University Mission Statement: NSU is a responsive, student-oriented institution that is committed to the creation, dissemination, and acquisition of knowledge through teaching, research, and service. The University maintains as its highest priority excellence in teaching in graduate and undergraduate programs. Northwestern State University prepares its students to become productive members of society and promotes economic development and improvements in the quality of life of the citizens in its region. College of Nursing and School of Allied Health Mission Statement: NSU CONSAH serves the people of Louisiana and in so doing improves the health of its citizens while advancing the mission of Northwestern State University through excellence in accessible undergraduate, graduate, and continuing education programs that are designed to assist individuals in achieving their professional goals as responsible and contributing members of their profession and society. School of Allied Health Mission Statement: The SAH at NSU is dedicated to providing high quality undergraduate and graduate programs that prepare individuals for a variety of professional healthcare roles and to be conscientious, contributing members of their profession and society. BSRS Mission The mission of the Radiologic Sciences Program is to provide students with advanced knowledge and skills through guided experiences and clinical practice that culminates in professional radiologic technologists becoming an integral part of the healthcare community and society. Bachelor of Science in Radiologic Sciences Purpose and Objectives: BSRS Program Purpose To provide students with the education and skills to function as an integral part of the health care community and the opportunity for advancement in the allied health professions. To provide opportunities which will enhance the development of roles in the radiologic sciences professions To provide a foundation for radiologic science professionals to become lifelong learners and to strive for continued professional growth BSRS Program Objectives Graduates of the BSRS program should be able to: Perform quality radiographic procedures. Develop assessment skills of a radiographer. Evaluate a clinical situation and perform accordingly using critical thinking skills. Propose a plan to respond to imaging department scenarios. Demonstrate service to the profession and the community. Integrate adherence to professional behaviors. Develop oral and written communication skills.

20 SLO 1: Students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their discipline. Benchmark Tool Benchmark Results Met/ Unmet Analysis/ Action RADS 4711 Learning Unmet for Life rubric (Q.1) Students will Correlate prior life experiences with academic knowledge and experiences. Principle 2 90% of students will score at least 3/4 on benchmark N 40 Met 35 % 87.5 Mean 2.9 Range 2-4 Five students scored a 2 (Basic) on this item. During the evaluations, all students were encouraged to review previous assignments and textbooks to become better prepared for their clinical rotations. 2 Connect concepts through an interdisciplinary perspective. Principle 1 Adapt and implement previously learned knowledge and skills to new contexts, situations, or scenarios. Principle 6 Communicate effectively. Principles 2 & 6 Expand sense of future self through reflection on participation in experiential learning processes. Principle 4 RADS 4711 Learning for Life rubric (Q.2) RADS 4711 Learning for Life rubric (Q.3) RADS 4711 Learning for Life rubric (Q.4) RADS 4711 Learning for Life rubric (Q.5) 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark N 40 Met 37 % 92.5 Mean 3.2 Range N 40 Met 37 % 92.5 Mean 3.3 Range N 40 Met 39 % 97.5 Mean 3.5 Range N 40 Met 34 % 85 Mean 2.8 Range 2-4 Met Met Met Unmet Three students scored a 2 (Basic) in this item. Overall, however, the majority of students (92.5%) scored a satisfactory mark in this category. Three students scored a 2 (Basic) in this item. Overall, however, the majority of students (92.5%) scored a satisfactory mark in this category. Students scored the highest in this category. Only one students scored a 2 (Basic) in this item. Overall, however, the majority of students (97.5%) scored a satisfactory mark in this category. Six students scored a 2 (Basic) on this item. During the evaluations, all students were encouraged to review previous assignments and textbooks to become better prepared for their clinical rotations. Students were also encouraged to consider future options with regard to their clinical sites, and the importance of performing well.

21 3 Demonstrate professional characteristics and behaviors. Principles 2 & 6 RADS 4711 Learning for Life rubric (Q.6) 90% of students will score at least 3/4 on benchmark. N Met % Mean Range Unmet Four students scored a 2 (Basic) on this item, while one student scored a 1 (Developing). During the evaluations, all students were encouraged to remember that the internship experience was essentially a job interview, and as such, were expected to model professional behaviors at all times.. SLO 2: Students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. Benchmark Tool Benchmark Results Met/ Unmet Analysis/ Action RADS 4711 Learning Met for Life rubric (Q.1) Students will Communicate effectively using appropriate conventions of language and correct format(s). Principles 2 & 6 90% of students will score at least 3/4 on benchmark N 40 Met 37 % 92.5 Mean 3.2 Range 2-4 Two students scored a 2 (Basic) on this item, while the majority of students scored a 3 (Mastery). During the evaluations, all students were encouraged to continue refining their communication strategies and to be familiar with specific terminology used in their specific discipline. Connect prior learning to changes that are a direct result of the experiential learning process. Principle 4 Revisit prior learning to apply knowledge and skills in new and innovative ways. Principles 4 & 6 RADS 4711 Learning for Life rubric (Q.2) RADS 4711 Learning for Life rubric (Q.3) 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark N 40 Met 34 % 85 Mean 2.8 Range N 40 Met 31 % Mean 2.7 Range 1-4 Unmet Unmet Eight students scored a 2 (Basic) in this item. The remainder scored a 3 or 4. Students were encouraged review their past lessons in various specialties to encourage growth. One student scored a 1 (Developing), six students scored a 2 (Basic), and the remainder scored a 3 or 4. Faculty spent extended time with each student to encourage thinking outside of the box ideas.

22 4 Assess what they have learned about themselves as members of a broader community. Principle 7 Assess what they have learned about themselves as individuals. Principle 7 RADS 4711 Learning for Life rubric (Q.4) RADS 4711 Learning for Life rubric (Q.5) 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark. N Met % Mean Range N Met % Mean Range Unmet Unmet This section was scored almost identically to the previous question. Again, students were encouraged to think about their roles not only in their specialties, but also within the larger healthcare community. Four students scored a 2 (Basic) on this item, with the remainder scoring a 3 or 4. Faculty felt that this item was being met, but that there was still room for improvement.

23 1 BSRS Learning for Life Final Evaluations Northwestern State University Mission Statement: NSU is a responsive, student-oriented institution that is committed to the creation, dissemination, and acquisition of knowledge through teaching, research, and service. The University maintains as its highest priority excellence in teaching in graduate and undergraduate programs. Northwestern State University prepares its students to become productive members of society and promotes economic development and improvements in the quality of life of the citizens in its region. College of Nursing and School of Allied Health Mission Statement: NSU CONSAH serves the people of Louisiana and in so doing improves the health of its citizens while advancing the mission of Northwestern State University through excellence in accessible undergraduate, graduate, and continuing education programs that are designed to assist individuals in achieving their professional goals as responsible and contributing members of their profession and society. School of Allied Health Mission Statement: The SAH at NSU is dedicated to providing high quality undergraduate and graduate programs that prepare individuals for a variety of professional healthcare roles and to be conscientious, contributing members of their profession and society. BSRS Mission The mission of the Radiologic Sciences Program is to provide students with advanced knowledge and skills through guided experiences and clinical practice that culminates in professional radiologic technologists becoming an integral part of the healthcare community and society. Bachelor of Science in Radiologic Sciences Purpose and Objectives: BSRS Program Purpose To provide students with the education and skills to function as an integral part of the health care community and the opportunity for advancement in the allied health professions. To provide opportunities which will enhance the development of roles in the radiologic sciences professions To provide a foundation for radiologic science professionals to become lifelong learners and to strive for continued professional growth BSRS Program Objectives Graduates of the BSRS program should be able to: Perform quality radiographic procedures. Develop assessment skills of a radiographer. Evaluate a clinical situation and perform accordingly using critical thinking skills. Propose a plan to respond to imaging department scenarios. Demonstrate service to the profession and the community. Integrate adherence to professional behaviors. Develop oral and written communication skills.

24 2 SLO 1: Students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their discipline. Benchmark Tool Benchmark Results Met/ Unmet Analysis/ Action RADS 4711 Learning Met for Life rubric (Q.1) Students will Correlate prior life experiences with academic knowledge and experiences. Principle 2 90% of students will score at least 3/4 on benchmark N 40 Met 40 % 100 Mean 3.1 Range 3-4 While the overall percentage increased slightly, all students scored a minimum of a 3 (Mastery), which was considered an improvement from the midterm evaluation. Connect concepts through an interdisciplinary perspective. Principle 1 Adapt and implement previously learned knowledge and skills to new contexts, situations, or scenarios. Principle 6 Communicate effectively. Principles 2 & 6 Expand sense of future self through reflection on participation in experiential RADS 4711 Learning for Life rubric (Q.2) RADS 4711 Learning for Life rubric (Q.3) RADS 4711 Learning for Life rubric (Q.4) RADS 4711 Learning for Life rubric (Q.5) 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark N 40 Met 36 % 90 Mean 3.0 Range N 40 Met 40 % 100 Mean 3.4 Range N 40 Met 39 % 97.5 Mean 3.5 Range N 40 Met 37 % 92.5 Mean 3.1 Met Met Met Met While more students were evaluated as a 4 (Advanced), it was curious that there were more students (five) who scored a 2 (Basic) in this item. Overall, however, the majority of students (90%) scored a satisfactory mark in this category. All students were evaluated as being either Advanced or Mastery in their abilities to implement previously learned knowledge in their current situations. One student scored a 2 (Basic) in this item. Overall, however, the majority of students (97.5%) scored a satisfactory mark in this category. Three students scored a 2 (Basic) on this item, which was an improvement from the previous evaluation.. Students continued to be encouraged to consider future options with regard to their options of imaging specialties.

25 3 learning processes. Principle 4 Range 2-4 Demonstrate professional characteristics and behaviors. Principles 2 & 6 RADS 4711 Learning for Life rubric (Q.6) 90% of students will score at least 3/4 on benchmark Met Much improvement was made in this category, as all N 40 students scored a 3 or 4 in this section. Students were Met 40 reminded to model professional behaviors as they % 100 entered into their professions. Mean 3.3 Range 3-4 SLO 2: Students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. Benchmark Tool Benchmark Results Met/ Unmet Analysis/ Action RADS 4711 Learning Met for Life rubric (Q.1) Students will Communicate effectively using appropriate conventions of language and correct format(s). Principles 2 & 6 90% of students will score at least 3/4 on benchmark N 40 Met 40 % 100 Mean 3.3 Range 3-4 All students scored either a 3 or 4 in this category. Faculty continued to stress the importance of effective communication as the students entered into their chosen professions. Connect prior learning to changes that are a direct result of the experiential learning process. Principle 4 Revisit prior learning to apply knowledge and skills in new and innovative ways. Principles 4 & 6 RADS 4711 Learning for Life rubric (Q.2) RADS 4711 Learning for Life rubric (Q.3) 90% of students will score at least 3/4 on benchmark. 90% of students will score at least 3/4 on benchmark N 40 Met 36 % 90 Mean 3.1 Range N 40 Met 34 % 85 Mean 2.9 Range 2-4 Met Unmet Four students scored a 2 (Basic) in this item. The remainder scored a 3 or 4. Students were encouraged review their past lessons in various specialties to encourage growth after graduation. While six students scored a 2 (Basic), no students scored a 1, which was an improvement from the midterm evaluation. Faculty spent extended time with each student to encourage thinking outside of the box ideas as they moved into their careers.

26 4 Assess what they RADS 4711 Learning 90% of students 2017 Unmet Seven students were still considered to be basic with for Life rubric (Q.4) will score at least N 40 regard to their assessment of themselves within the 3/4 on benchmark. have learned about themselves as members of a broader community. Met 33 % 82.5 Mean 2.7 Principle 7 Range 2-4 broader community. Faculty continued to stress the importance of thinking in much broader terms regarding the student s role in the workfoce. Assess what they RADS 4711 Learning 90% of students 2017 Unmet The average score for this item was much higher than the have learned for Life rubric (Q.5) will score at least N 40 midterm evaluation. No further action was deemed about themselves 3/4 on benchmark. necessary. as individuals. Met 40 Principle 7 % 100 Mean 3.1 Range 3-4 Summary of Decisions I. Students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their discipline. All six measures for this for this outcome were met. Faculty felt there was a marked improvement in the behaviors of students since the midterm evaluations. Faculty also felt that students had the experience necessary to be successful in this internship project, but needed to be reminded that success was also dependent on student preparation. II. Students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. Two measures met the benchmark and three did not. For the three that were unmet, faculty realized that there was some work to do in finding ways to encourage students to think about themselves and their situations differently. While there was no instruction for the pilot program, the new revised QEP should address these issues more fully.

27 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Pilot Program. Early Childhood Education BS Internship Assessment Louisiana Scholars' College of Education Prepared by: Michelle Fazio Brunson Date: June 10, 2017 Approved by: Date: Student Learning Outcomes (SLO). SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. (Aligned with Principle 1-Intention and Principle 2-Preparedness and Planning) SLO 2. During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. (Aligned with Principle 4-Reflection and Principle 6-Monitoring and Continuous Improvement) Measure. Successfully completing a nine-hour residency semester is a graduation requirement for the college's Bachelor of Science with a major in Early Childhood Education (3101). During this field-based residency experience, candidates spend all day in the classroom for the duration of the semester. They are responsible for writing and teaching lesson plans, assessing young children, and documenting the children s growth. Candidates get feedback on lessons, and they reflect on their experiences as they add to their professional teaching portfolio. At the end of the semester, candidates were asked to complete a survey addressing SLO 1 and SLO 2 of the QEP. Methodology. In Spring 2017, 2 candidates completed an Early Childhood residency. Both candidates completed the QEP survey before graduation. Their responses are compiled below. Target. The goal was for 100% of candidates to complete the survey. Finding. Target Met. Q1: Describe how your prior knowledge influenced your internship. (100 words or less).

28 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Candidates indicated that they used prior knowledge in their field experience while writing lesson plans. Q2: What transferable skills did you bring to the internship? (Select all that apply.) Candidates indicated that they brought these skills into their internship: Time management, Plan/organize events, Collaboration, Oral communication, Written communication, Work independently, Follow professional style guidelines Problem solving, and Listening. Q3: What skills did you improve during the internship? (Select all that apply.) Candidates indicated that they improved these skills in internship: Time management, Problem solving, Delegate/instruct others, Plan/organize events, Listening, Collaboration, Oral communication, Written communication, Presentation, Work independently, Creative & critical thinking, Follow professional style guidelines Q4: What new skills did you develop during the internship? (Select all that apply.) Candidates indicated that they developed these skills during internship: Plan/organize events, Listening, Oral communication, Written communication, Presentation, and Time management Q5: What skills do you still feel you need to improve after the internship? (Select all that apply.) Candidates indicated that they still need to improve skills relating to: Time management, Collaboration, and Analyze data/audit. Q6: In manners of professional communication, describe the ways in which you have communicated effectively during your internship? (Select all that apply.) Candidates indicated that they effectively communicated through: , Written, Oral, Reflection, Telephone, and Interview pathways. Q7: Rate your professional communication skills as you began the internship as either "Needs Improvement", "Average", "Excellent." 100% of candidates self-scored Average in the following at the beginning of their internship: Time management, Ability to plan/organize events, Problem solving, Manage money/budgets, Written communication, and Presentation, and Listening 50% of candidates self-scored Excellent and 50% self-scored Average in the following at the beginning of their internship: Ability to delegate/instruct others, Collaboration, Oral communication, and Ability to

29 give/receive feedback Quality Enhancement Program Pilot Program Assessment Period Spring % of candidates self-scored Average and 50% self-scored Needs Improvement in the following at the beginning of their internship: Analyze data/audit. Q8: Rate your professional communication skills as you complete your internship. 100% of candidates self-scored Average in the following at the end of their internship: Time management, Ability to plan/organize events, Analyze data/audit, Written communication, Listening, and Presentation. 100% of candidates self-scored N/A in Manage money/budgets at the end of their internship. 50% of candidates self-scored Excellent and 50% self-scored Average in the following at the end of their internship: Ability to delegate/instruct others, Problem solving, Collaboration, Oral communication, and Ability to give/receive feedback Q9: Describe one instance in which you feel you demonstrated the most growth in your communication skills or learned the most about how to communicate as a professional during your internship. (100 words or less.) Candidates felt they grew the most in presenting my student learning target results at the end of the semester and listening to others giving feedback. Q10: Select your major from the list below: Both candidates are Early Childhood Education majors. Q11: Describe one way in which your internship helped you relate your major to another discipline (100 words or less). One candidate indicated that that the internship gave her a year long experience in a third grade classroom. The other candidate answered N/A. Q12: Describe two of your professional strengths (25 words or less) Candidates indicated their professional strengths were: Classroom Management, Communication, and Planning/Prep. Q13: Describe two of your professional weaknesses (25 words or less). Candidates indicated that their professional weaknesses were: Technology, Teaching ESL Learners, Time management, and Classroom management.

30 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Q14: What is your plan to address the professional weaknesses (50 words or less)? Both candidates plan to address their weaknesses through Professional development. Q15: As you reflect on your internship, how will you continue to learn and grow professionally (50 words or less)? The candidates plan to engage in Professional Development and to continue to improve classroom management and planning skills. Q16: How did you personally and professionally benefit from participation in the internship? (100 words or less) One candidate learned from other teachers and gained great skills. The other candidate left this question blank. Q17: How do you think the internship will help you impact the greater community of which you are a member? (100 words or less). Both candidates plan to give back by teaching. Q18: How do you feel this process could be improved to benefit future participants? (100 words or less) One candidate quoted, It's a great experience being able to be in class for a year and watching students grow. The second candidate suggested more professional development with technology. Q19: Select three words that best describe your reflections about your experiences with the internship. Both candidates felt: Excited, Enthused, and Prepared. Analysis. Data gleaned from the pilot study can be used to inform the next phase of data collection. First, while both candidates completed the survey, there were only two, generalizing findings isn t possible. Second, while the candidates answered almost all of the questions, some of their answers were short. In the next phase, we ll know to stress the importance of providing quality, substantive answers. This emergent data can also be used to inform the field-based residency experience next year. For example, the candidates indicated that they still needed to develop skills relating to Time management, Collaboration, and Analyze data/audit. This could be addressed through professional development. Second, candidates indicated that their professional weaknesses were: Technology, Teaching ESL Learners, Time management, and Classroom management. These could also be addressed through professional development sessions.

31 Quality Enhancement Program Pilot Program Decision / Recommendations. Assessment Period Spring 2017 Next semester, the survey needs to be built into the residency syllabus so that all candidates complete the survey before graduation. It would be helpful to take all candidates to a lab to complete the survey at the same time. Finally, survey completers need to be tracked so we know who has and has not completed the survey so we can reach out to specific candidates if necessary.

32 Northwestern State University Quality Enhancement Plan (QEP) Pilot Program Assessment Report Pilot Program: Elementary Education Prepared by: Ramona A. Wynder Date: July 5, 2017 A. Pilot Program Description The Elementary Education undergraduate program was selected for the Northwestern State University s Learning for Life: Quality Enhancement Plan (QEP) Experiential Learning pilot. Thus, during the pilot year, six Elementary Education teacher candidates participated in a year-long, field-based residency in local schools, working under highlyqualified Elementary teachers. Over the course of the two-semester residency, teacher candidates had opportunities to observe accomplished teaching; plan and teach lessons; and analyze and reflect on their own practice and student learning. This capstone experience exposed teacher candidates to the realities of the classroom and helped them develop the habits of mind and practices of effective teachers. B. QEP Assessment Method The goal of the Learning for Life is to prepare candidates to transfer theory into practice as they transition from Northwestern State to a career or advanced study in graduate schools or professional schools. In order to measure achievement of this goal and to ensure each degree program is effective and relevant, a thorough, systematic assessment plan was implemented during the pilot year. Survey instruments and rubrics were developed to assess the following Student Learning Outcomes (SLOs), aligned with the National Society of Experiential Education s Eight Principles of Best Practice for All Experiential Learning Activities (1998): 1. SLO 1 During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. Benchmarks: Students will Correlate prior life experiences with academic knowledge and experiences. (Principle 2) Connect concepts through an interdisciplinary perspective. (Principle 1) Adapt and implement previously learned knowledge and skills to new contexts, situations, or scenarios. (Principle 6) Communicate effectively. (Principles 2 and 6) Expand sense of future self through reflection on participation in experiential learning process. (Principle 4) Demonstrate professional characteristics and behaviors. (Principles 2 and 6) 2. SLO 2 During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection.

33 Benchmarks: Students will Northwestern State University Quality Enhancement Plan (QEP) Pilot Program Assessment Report Communicate effectively using appropriate conventions of language and correct format(s). (Principles 2 and 6) Connect prior learning to changes that are a direct result of the experiential learning process. (Principle 4) Revisit prior learning to apply knowledge and skills in new and innovative ways. (Principles 4 and 6) Assess what they have learned about themselves as members of a broader community. (Principle 7) Assess what they have learned about themselves as individuals. (Principle 7) At the end of the pilot program year, the director of Field Experience and Clinical Practice evaluated the six resident teachers professional readiness using the QEP Rubrics for SLO 1 and 2. The rubrics for both SLOs contain four levels of assessment: 4 is Advanced; 3 is Mastery; 2 is Basic; and 1 is Developing. The goal was for 50% of candidates to score at a Level 3 or Level 4 on 50% of the items contained on the rubric. The benchmark was for 50% of candidates in each pilot program to have an average score of 3 or greater. The findings are presented below. C. Assessment Results Table 1. Percent of Resident Teachers Scoring at Each Performance Level on SLO 1 Rubric by Indicator n=6 Advanced (4) Indicators: 1. Correlate prior life experiences with academic knowledge and experiences 2. Connect concepts through an interdisciplinary perspective 3. Adapt and implement previously learned knowledge and skills to new contexts, situations, or scenarios. 4. Communicate effectively 5. Expand sense of future self through reflection on participation in experiential learning process 6. Demonstrate professional characteristics and behaviors Mastery (3) Basic (2) Developing (1)

34 Northwestern State University Quality Enhancement Plan (QEP) Pilot Program Assessment Report Table 2. Percent of Resident Teachers Scoring at Each Performance Level on SLO 2 Rubric by Indicator n=6 Advanced (4) Indicators: 1. Communicate effectively using appropriate conventions of language and correct format(s 2. Connect prior learning to changes that are a direct result of the experiential learning process 3. Revisit prior learning to apply knowledge and skills in new and innovative ways 4. Assess what they have learned about themselves as members of a broader community 5. Assess what they have learned about themselves as individuals Table 3. Resident mean scores by SLO Rubrics Mastery (3) Basic (2) Developing (1) n=6 SLO 1 mean out of 4 SLO 2 mean out of 4 Total mean score Resident Resident Resident Resident Resident Resident Group mean D. Analysis of Assessment Data On SLO Rubric 1, the pilot program group exceeded the QEP goal by scoring at the Mastery level or better on all indicators (Table 1). The data further revealed a strength on Indicator 6 (professionalism), with 83% of candidates earning a rating of Advanced. However, on Indicator 5 (reflection), 50% of residents received a rating of Basic, indicating a need for improvement on this skill. On Indicator 5 (self-awareness) of SLO Rubric 2, the results were the same 50% of the group earned a score of Basic (Table 2). Nevertheless, the group reached the goal for SLO Rubric 2 as well. Data reported in Table 3 have been calculated to show mean scores for each resident teacher on both rubrics and total mean scores for the group. Disaggregate data indicates the goal of the group having an average score of at least 3 was met. The target of at least 50% of the

35 Northwestern State University Quality Enhancement Plan (QEP) Pilot Program Assessment Report residents earning a mean rating at the Mastery level was also reached. E. Interpretation of Assessment Data The data showed Elementary Education teacher residents met, and in some cases, surpassed the benchmarks set by the QEP: Learning for Life governing committee. Data gleaned from the SLO Rubrics help affirm the residents possess the knowledge, skills and dispositions expected of first-year teachers. Although the data revealed no significant areas of concern, the residents ability to reflect on their personal and professional growth is considered a skill in need of refinement. This finding is commensurate with other data collected and analyzed by program faculty over the course of the resident year. Consequently, in fall 2017, faculty will participate in professional development to increase their capacity to foster residents growth and development into reflective practitioners and will also redesign reflection instruments used in the Elementary program. Faculty are committed to ensuring teacher candidates are profession-ready when they complete the program and assume roles as full-time teachers. Reference:

36 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Pilot Program. HHP Undergraduate Internship Program Prepared by: John E. Dollar Date: July 6, 2017 Approved by: Date: Student Learning Outcomes (SLO). SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines. (Aligned with Principle 1-Intention and Principle 2-Preparedness and Planning) SLO 2. During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflection. (Aligned with Principle 4-Reflection and Principle 6-Monitoring and Continuous Improvement) Measure. A 12-hour senior internship (HP 4200) is a graduation requirement for the Bachelor of Science degree in Health and Exercise Science (HAES). This degree program is hosted in the Department of Health and Human Performance in the Gallaspy Family College of Education and Human Development. The HAES internship requirement is comprised of 12 hours of academic credit for a minimum of 500 hours of contact time for the student intern, exercised through a work setting of approximately 40 hours per week for the 15-week semester. The internship site is selected by the student, with the approval of the university supervisor (department head), and the internship is governed by a contract between the university, the student and the internship site. Students complete and submit a variety of assignments and projects throughout the semester of the internship. Students prepare for the internship experience through a graded, pre-internship seminar (HP 4180) class the semester prior to the internship. Performance guidelines and assignment expectations are reviewed during that time. Students receive a mid-term and a final evaluation from the respective site supervisor, as well as grades from the university supervisor for individual assignments submitted throughout the semester of the internship. Once completed, the students meet with the university supervisor for an exit interview, prior to graduation. Methodology. In the Spring semester of 2017, 11 students completed their internship in HAES. Students were assessed two times: at the mid-term evaluation from the site supervisor, and at the point of the exit interview with the university supervisor. At the exit interview, the department head independently rated the student according to the rubrics for SLO1 and SLO2 (Learning for Life (LfL), pp ), and based solely on the performance and information given during the face-to-face interview. The university supervisor asked direct questions, addressing rubric benchmarks when the

37 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 information was not apparent from the student's presentation of their portfolio, and / or the information did not arise independently in the exit interview document, which was completed by the student, prior to the meeting. The responses for each benchmark were averaged for each student. No ratings were omitted, and all ratings were included in the averages. Target. The goal of the program is for at least 50% of students to perform at level three (3) Mastery or level four (4) Advanced on each rubric item for each SLO. The overall benchmark is for 50% of students to have an average score of 3 or greater (LfL, p. 41). Finding. Target Met. HAES students exceeded this goal (Table 1). The worst performance was on SLO 1 (During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of entry-level professionals in their disciplines), where 3 students (27%) averaged lower than 3.0 on Benchmark 1 (Students will communicate effectively using appropriate conventions of language and correct format(s)). The second highest number of students (2=18%) who averaged below a 3.0 occurred on Benchmark 2 (Students will connect concepts through an interdisciplinary perspective). None of the remaining benchmarks for SLO1 or SLO2 were missed by more than a single student. Thus at least seven of the eleven (64%) of the HAES student interns performed at 3.0 average score or higher for each rubric item for each SLO. Table 1. Number of Pilot Program Undergraduate Internship Participants (n=11) Scoring below 3.0 Note: N = 11 SLO1 benchmark # SLO2 freq benchmark # freq % 1 0 0% % 2 1 9% 3 1 9% 3 1 9% 4 1 9% 4 0 0% 5 1 9% 5 1 9% 6 0 0% Students were also evaluated on their average scores for all rubric items. The average of this score was 3.34 (SD = 0.526, N = 11). Again, the benchmark was exceeded; 64% of the population had a mean score above 3.0. The 95% confidence intervals for the mean score for each of these 7 students was also above 3.0.

38 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Analysis. The pilot study revealed a number of issues with the application of the rubric. First, while the rubric was very appropriate for courses in which a reflection component was administered (weekly and in the discussion board assignments), we want to continue to administer this form. However, the reflection components of the internship assignments could be enhanced. There is a need to draw out from the student how previous courses and learning experiences specifically increase the effect of the internship. This enhancement could be accomplished through tweaking the rubric to include specific skills and dispositions unique to the HAES internship setting. Secondly, the application of each benchmark rubric to each internship experience was carried out by a single university HAES supervisor. This assessment process of the rubric to the internship experiences, should be completed by two or more HAES evaluators. There should be training provided to faculty evaluators to ensure consistent application of the rubric benchmarks as they relate to the internship experiences. We should also relate consistent statistics in support of the related outcomes. If the data are inconsistent, then the statistics provide the wrong results. Decision / Recommendations. We recommend faculty training / development for the application of the benchmarks of the rubric to the internship experiences. This may require verbiage refinement to the rubric(s). We also It is recommended the rubric be translated to point out skills and dispositions that occur specific to the internship setting.

39 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Pilot Program. Hospitality Management and Tourism Prepared by: Valerie Salter Date: June 27, 2017 Approved by: Date: Student Learning Outcomes (SLO). SLO 1. During the capstone experiential learning course(s), students will demonstrate the knowledge, skills, and dispositions expected of an entry level professional in their discipline. SLO 2. During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop new applications of knowledge based on these reflections. Measure. A twelve credit hour, 400 clock hour senior field experience is a graduation requirement for the college's Bachelor of Science with a major in Hospitality and Tourism and a concentration in Travel and Tourism (135C), Hospitality Services (135A), and Culinary Arts (135D). The twelve credit hours may be taken in one semester or divided into two six hour sections over two semesters. Students enrolling in the required capstone course HMT 4500: Field Experience ( internship ) are required to submit weekly documentation that includes a summary of a peer-reviewed journal article relating to their current job role and an in-depth reflection of the work they performed during the week including knowledge and experience gained, evaluation of their experiences that week, how they would approach situations differently based on their experiences, and what they learned in previous courses that have prepared them for their experiences that week. Additionally, students are required to complete a special project in which they take on additional responsibility outside of their regular role. Methodology. In Spring 2017, six students completed their field experiences and comprehensive portfolios. Cooperating employers were required to complete a mid-term and final evaluation assessing the student s professionalism, knowledge and skills.

40 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Students were assessed once at the end of the semester after they had completed their required clock hours and turned in their comprehensive portfolios. Students portfolios are graded by the following criteria and weights with a possible 100 points: (25%) Supervisor s Evaluation; (15%) Weekly Logs; (10%) Weekly Article Summaries; (20%) Special Project; and (30%) Professional Notebook For the assessment of SLO 1, students were expected demonstrate their competencies to their employers that meet or exceed the expectations of entry-level professionals in the hospitality industry. The evaluation that employers completed at mid-term and at the end of the semester are attached to this document. If students received an outstanding or good rating on a minimum of 16 out of the 20 competencies on the final evaluation, they receive points. If they received an outstanding or good on a minimum of 14 of the competencies, they received points. If they received outstanding or good on less than 14 competencies, they received 0-13 points. For the assessment of SLO 2, students were expected to provide an in-depth analysis of their weekly experiences and how those experiences related to the knowledge they gained from course work prior to their field experience. The weekly reflection log is attached to this document. The weekly reflection logs are 15% of the portfolio grade. Target. The goal of the program is for at least 75% of students to achieve an 70% or above on both SLO 1 and 2. Finding. Target Met. Hospitality Management and Tourism students exceeded this goal. The worst performance was on SLO 2 (During the capstone experiential learning course(s), students will reflect critically to link theory with practice and develop applications of knowledge based on the reflections), where 1 student (14%) averaged lower than 70% on SLO 2. The average for the other students for SLO 2 was 90%. Two of the students out of the seven could not provide supervisor evaluations due to one student being employed at Disney, and the other student continuing the length of her internship since the event she is assisting with has not occurred yet. There are student artifacts and copies of the evaluations attached to this document. Student Scores for Final Evaluation Provided by Supervisor/ QEP Rubric Benchmarks S= 20/25 Mastery B= 25/25 Advanced T= 25/25 Advanced M= 25/25 Advanced A1= None N/A A2= 20/25 Mastery

41 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 Analysis. The pilot study revealed a few minor of issues with the application of the rubric. The rubric very closely correlates with the components for the portfolio used for the pilot study. However for SLO 1, there may be some adjustments needed to the supervisor s evaluation form to fully incorporate all of the benchmarks listed in the rubric. For SLO 2, there may need to be additional reflection questions added to the weekly log in order to fully explore and document the connections between theory and practice. The changes needed to these documents will require the supervisor and students to go more in depth than they have in previous documentation. Hospitality Management faculty will need further training and communication about utilizing the rubrics to grade the portions of the comprehensive portfolio for HMT With the right training and implementation, this rubric could also be used for other activities and assignments in different capstone courses within the degree program. Further, the rubrics may need some minor changes reflecting faculty input as it relates to activities and assignments that are already in place. Decision / Recommendations. Hospitality Management and Tourism needs to make minor changes to the supervisor evaluation form and the weekly log documentation for HMT Additionally, the faculty need to go through other documentation for the course and evaluate them based on the benchmarks indicated in the provided rubrics. This may reveal that minor changes may need to be suggested to reflect more in-depth responses and analysis from the students. HMT faculty will need to be trained and clearly communicate with each other about the use of rubrics to grade HMT 4500 portfolios. For SLO 1, faculty may want to match the skills provided on the supervisor evaluation form with the benchmarks on the rubric. Additionally, students will need to be oriented with the required documentation and understand faculty expectations on the in-depth reflection and responses needed to achieve a passing score.

42 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 NORTHWESTERN STATE UNIVERSITY HMT 4500 Field Experience NON-CONFIDENTIAL FINAL EVALUATION Student: Cooperating Industry: University Supervisor: This non-confidential form was designed to serve as a learning experience for the student. As such it would be very desirable for each supervisor to discuss the rating with the intern about halfway through the experience and again at the end. In addition to ratings, comments would be helpful. Thank you. INTEREST: Does the studenttake the internship seriously and appreciate its importance? Date Evaluation is Due: Semester: Industry Supervisor: Rating Key: O-Outstanding G-Good I-Improvement Needed N-No Basis For Judgement Comments show sustained interest in routine duties? ask questions and discuss things not understood? make suggestions of ways to function more fully in the organization? PERFORMANCE: Does the student - cooperate fully? show ability to learn new skills, methods and ideas? comprehend functions and responsibilities? complete assignments and meet deadlines? obtain necessary information for effective performance?

43 JUDGEMENT: Does the student - exercise sound judgement and understand explanations easily? Quality Enhancement Program Pilot Program SOCIAL ABILITY: Does the student - practice punctuality? Assessment Period Spring 2017 appear neat and clean? show tact and courtesy? show poise and self-reliance? TEMPERAMENT: Does the student - exercise self-control? get along well with personnel? INTELLECTUAL HONESTY: Does the student - appear to be receptive to suggestions? appear to be objective, concerning previous ability and experiences? EFFECTIVENESS: Does the student - perform effectively in working with the public served? comprehend the organization's function and purpose? Comments concerning student's performance: Please rate this student as to his/her prospect for success in the industry. Excellent (A) Good (B) Fair (C) Poor (D) Based on this student's performance, would you be willing to participate in the program again? Yes No Other Evaluated by: Signature Title:

44 Quality Enhancement Program Pilot Program Assessment Period Spring 2017 STUDENT FIELD EXPERIENCE WEEKLY LOG NAME: COOPERATING EMPLOYER: WEEK OF: FIRM: HOURS WORKED: TOTAL ACCUMULATED HOURS: DESCRIPTION OF ACTIVITIES: (Be thorough, including your opinion toward the value of time spent in comparison with experience and knowledge gained and any other pertinent information.) Weekly log is due on Monday following the week completing work. 1. Describe work performed: 2. Knowledge, experience and/or skills gained in relation to your educational objectives: 3. Evaluation of experience: 4. Based on 1 or 2 of your unique experiences this week, if you were to encounter the same situation again, what would you have done differently? 5. Based on your reflection in the previous question, what did you learn during your course work in HMT classes that prepared you for these experiences? 6. Comments:

45 Quality Enhancement Program Pilot Program Assessment Period Spring 2017

46 Quality Enhancement Program Pilot Program Assessment Period Spring 2017

47 Quality Enhancement Program Pilot Program Assessment Period Spring 2017

48 Quality Enhancement Program Pilot Program Assessment Period Spring 2017

49 Quality Enhancement Program Pilot Program Assessment Period Spring 2017