OREGON TECH - OHSU CLINICAL LABORATORY SCIENCE PROGRAM 2013-2014 Annual Assessment Report Submitted by Cara Calvo, MS, MT(ASCP)SH CLS Department Chair, Program Director, and Assessment Coordinator 1/16/2014 ABSTRACT: An illustrated and concise report of the assessment activities carried out by the faculty of Department of Clinical Laboratory Science, Wilsonville, Oregon. Every attempt has been made to report objective, accurate information. Any errors in reporting are not intentional, and the author welcomes correction when merited.
I. Introduction Clinical Laboratory Science (CLS) Program Assessment Report a. HISTORY & LOCATION: Established in 1933 by the Oregon Health and Science University (OHSU) in Portland, Oregon, the nationally accredited * Clinical Laboratory Science program is a universitybased, 3+1 program of study culminating in a BS in Clinical Laboratory Science. In 2001, administrative responsibilities for the program transferred to Oregon Tech through a master collaboration agreement between the two universities. Student diplomas identify both Oregon Tech and OHSU as the degree-granting institutions. In brief, Oregon s only baccalaureate CLS program retains the brand identity of OHSU with the administrative support of Oregon Tech. Today, the program is administered through the Department of CLS which resides on the Oregon Tech Wilsonville, Oregon campus. Here, students admitted to the last year of the degree program (professional year) take coursework that combines a rigorous competency-based science curriculum with community-sponsored clinical training. During the first four terms of the professional year, students complete course work in state-of-the-art-classrooms that include two well-equipped laboratory classrooms and an instrumentation room. Upon successful completion of the on-campus coursework, students are assigned to one or more program-affiliated laboratories to complete clinical training. During the 16-week clinical training period, students spend 40 hours per week applying knowledge and skills to perform a wide variety of testing in an accredited medical laboratory and to further develop discipline-specific competency under supervision of clinical instructors. Currently, the Department of CLS maintains affiliations with accredited laboratories in Oregon, Washington, Nevada, and Idaho. Program graduates are eligible to take the American Society for Clinical Pathology (ASCP) Medical Laboratory Scientist (MLS) national board certification examination and to pursue career opportunities in various laboratory settings including but not limited to medical, research, and public health. * The Clinical Laboratory Science professional program is accredited by the National Accrediting Agency for Clinical Laboratory Science (NAACLS), 5600 North River Road, Suite 720, Rosemont, Illinois 60018-5119. (773) 714-8880. b. ENROLLMENT TRENDS: During the six-year period 2007-2013, applications to the program more than doubled though the number of applications received in 2014 (January) was down from the previous year: 102 compared to 125 in 2013. The number of admitted students has also doubled from 25 (2007) to 50 (2013). The program admitted 48 (46 regular admission plus 2 EACLSP, Klamath Falls) students for the 2014-2015 year. c. RETENTION RATES: The numbers of admitted students graduating (retention rate) from the CLS program is excellent over the last six years for which records are available (2007 2013). During this time, 155 students graduated of the 164 that were admitted for a retention rate of 94.5%. d. RECENT NUMBER OF GRADUATES: 26 graduated 12/8/2012; 35 graduated 12/14/2013 e. EMPLOYMENT RATES & SALARIES: Based on results of graduate exit surveys (2009-2013) and reports from the most recent graduates (class of 2013) employment rates for program graduates are good. Approximately 78% (103/139) of graduates reported that they were employed within six months of graduation. As of July 2014, 26 of the 35 2013 graduates reported being employed as MLS. The starting annual salaries for the most recent graduates (2013) is reported to be $ 48,901 - > $52,000. f. PASS RATES ON NATIONAL BOARD EXAM: The pass rates for first time examinees on the American Society for Clinical Pathology (ASCP) board exam for Medical Laboratory Scientists is excellent over the last seven years 2007 2013. The averaged pass rate stands at 96.0 %. The pass rate for the most recent graduates (December 2013) stands at 94.1% Page 1 of 20
II. Clinical Laboratory Science (CLS) Program Assessment Report Program Purpose, Objectives, and Student Learning Outcomes Summary: Department and Advisory Board Review A meeting of the CLS Program Advisory Board scheduled for December 9, 2013 was cancelled due to adverse weather. A meeting will be re-scheduled for 2014. The CLS program faculty reviewed the program purpose, objectives, and student learning outcomes during the September 24, 2013 CLS Department Meeting and affirmed the following: Faculty & Staff Commitment We, the faculty and staff of the Oregon Tech OHSU CLS program, as professionals and educators, are committed to providing our students with experiences that prepare them to practice as scholastically accomplished and competent Medical Laboratory Scientists. To that end, we acknowledge our responsibility and pledge our commitment to: Demonstrate professionalism through our words and actions Provide knowledge-building, skill-developing experiences for all our students Create equal opportunity learning environments within which all our students are educated in an atmosphere of fairness and impartiality Foster students commitment to lifelong learning Endorse student participation in professional organizations Promote student scholarship and professional achievement Contribute to the ongoing development and growth of medical laboratory science pedagogy through faculty participation in scholarship, service, and outreach Program purpose, mission, objectives, and student learning outcomes (SLOs) Program Vision Our vision is that the Oregon Tech OHSU CLS program continue as a center of excellence in medical laboratory science education, graduating a Medical Laboratory Science workforce in demand. Program Mission The mission of the Oregon Tech OHSU Clinical Laboratory Science Program is to educate, train, and graduate professionally competent and ethical individuals, committed to life-long learning, and who are prepared to meet current and future workplace challenges in medical laboratory science. Program Goals The goals of the Oregon Tech OHSU CLS program are to: 1. Advance an innovative curriculum that meets current and emergent pedagogical and professional development needs of students 2. Identify, establish, and maintain partnerships with community medical laboratories that provide exceptional educational experiences 3. Provide learning experiences rich in opportunities that maximize every student s potential to achieve MLS career entry-level competencies 4. Graduate competent MLS that meet the workforce needs of Oregon and underserved regions of the nation 5. Contribute to the advancement of MLS pedagogy and growth of the profession Page 2 of 20
Student Learning Outcomes Upon completion of the Oregon Tech OHSU CLS a student will have had opportunity to acquire knowledge and skills, and develop professional attributes of a Medical Laboratory Scientist. Consequently, at the time of graduation, students will be able to demonstrate: 1) Competency to perform a full range of testing in the contemporary medical laboratory encompassing pre-analytical, analytical, and post-analytical components of laboratory services, including hematology, chemistry, microbiology, urinalysis, body fluids, molecular diagnostics, phlebotomy, and immunohematology; 2) Proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data; 3) Professional conduct, respecting the feelings and needs of others, protecting the confidence of patient information, and not allowing personal concerns and biases to interfere with the welfare of patients. 4) Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions. 5) Application of safety and governmental regulations and standards as applied to medical laboratory practice. 6) Effective communication skill to ensure accurate and appropriate information transfer. c. Noted assessment changes 2013-2014: Replaced SLO 5 with SLO 4 because 1) SLO 5 had been assessed every year for the last 4 years and the data indicates no problems, and 2) SLO 4 has been assessed only once in the last four years. Additionally, not only have the topics in SLO 4 (administrative skills, quality assurance, fiscal resource and human resource management, etc.) been a focus of discussion in our most recent Advisory Committee meeting (fall 2012), but the topics have gained spotlight attention within our profession; minor curriculum revisions, approved by CPC last year and implemented this year (2013-2014) provide better opportunity to assess SLOs 4 and 6 than existed prior to the revisions. The CLS degree program is jointly sponsored by Oregon Tech and Oregon Health and Science University. This year, the OHSU Assessment Council reviewed the program SLOs and issued its report 2/13/2014. The review board recommended that the descriptive paragraph shown on the program web page (http://www.oit.edu/wilsonville/academics/degrees/clinicallaboratory-science) be used as the Program s purpose statement. d. Other learning opportunities available to students in the OIT OHSU CLS program included the following: Several students attended a two-day parasitology workshop presented by Lynne Garcia, an internationally recognized authority on diagnostic parasitology, and sponsored by Asante Rogue Regional Medical Laboratory in May, 2014. Membership and participation as a student member in the Oregon branch of the American Society for Clinical Laboratory Science (ASCLS); o An Oregon Tech CLS student has been elected to serve as the Oregon student representative to the national ASCLS conference in Chicago, Illinois. July 29 August 2, 2014. Page 3 of 20
III. Clinical Laboratory Science (CLS) Program Assessment Report Three-Year Cycle for Assessment of Student Learning Outcomes (SLO) The CLS faculty/staff confirmed only the 2013-2014 year assessment plan listed in Table 1. Until the program advisory board meets, the program faculty have a chance to review the new NAACLS accreditation standards and determine the impact of the new NAACLS accreditation standards on a re-design of the program assessment plan, and the program completes its NAACLS re-accreditation cycle (self-study submitted April 2014), a final decision on definitive assessment SLO cycle is pending. Table 1 Assessment of Clinical Laboratory Science Program SLO by Year Student Learning Outcomes 1. Competency to perform a full range of testing in the contemporary medical laboratory encompassing preanalytical, analytical, and post-analytical components of laboratory services 2. Proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data 3. Professional conduct and ethical behavior, respecting the feelings and needs of others, protecting the confidence of patient information, never allowing personal concerns and biases to interfere with the welfare of patients 4. Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions 5. Application of safety and governmental regulations and standards as applied to medical laboratory practice 6. Effective communication skill to ensure accurate and appropriate information transfer Assessment of SLO by Year 13-14 X Direct X Direct Not Assessed 2013-2014 X Direct & Indirect Not Assessed 2013-2014 X - Direct Page 4 of 20
IV. Clinical Laboratory Science (CLS) Program Assessment Report Summary of 2013-2014 Assessment Activities Student Learning Outcome #1: Competency to perform a full range of testing in the contemporary medical laboratory encompassing pre-analytical, analytical, and post-analytical components of laboratory services, including hematology, chemistry, microbiology, urinalysis, body fluids, molecular diagnostics, phlebotomy, and immunohematology. SLO #1 required curricula listed in the new curriculum map (table A) on page 17. MEASURES OF SLO 1. DIRECT MEASURE: National Board Exam - at the end of SPRING term 2014, program faculty assessed professional program students competency to perform a full range of testing (SLO #1) using results from the American Society for Clinical Pathology (ASCP) Board of Certification (BOC) Medical Laboratory Scientist (MLS) exam. 2. DIRECT MEASURE: at the end of FALL term 2013, program faculty assessed professional program students competency to perform a full range of testing (SLO #1) using results from the clinical externship final exam results. Within the first three months of graduation (December 2013) 34 graduates took the ASCP BOC. The ASCP BOC certifies through examination only those individuals who meet the academic and clinical prerequisites and who achieve acceptable performance (> scaled mean score of 400) on the examination. The exam is criterion-referenced and constructed to measure the competencies, including knowledge, skills, and abilities, linked to the content areas of practice in a contemporary medical laboratory. These areas are Blood Bank (BBNK), Chemistry (CHEM), Hematology (HEMA), Immunology (IMMUN), Laboratory Operations (LO), Microbiology (MICR), and Urinalysis (UA). Also, in the fall term of 2013, 35 externship students took final exams after completing clinical rotation sections which included: a) chemistry/immunology, b) hematology/urinalysis, c) transfusion medicine, d) microbiology/serology, and e) specimen collection/safety. Students who did not pass an exam with a score of > 75% the first time were permitted a second attempt. SLO #1 ASSESSMENT DESIGN & DATA: assessment designs and results of the externship final exams and the ASCP BOC certification exams are shown in tables 2-5. Table 2 SLO #1 Assessment Design Performance Criterion Competently perform a full range of testing in the contemporary medical laboratory encompassing pre-analytical, analytical, and post-analytical components of laboratory services Assessment Method Direct: Externship Final Exams Direct: National Board Exam (D) Measurement Scale Relative (%) Scaled Mean Scores: total, & subtest, breakout Min. Acceptable Performance >95% of all examinees pass section exams with scores > 75% >95% of all 1 st time examinees pass with scores >400 Table 3 CLS 470 Externship Final Exam Outcomes per Rotation Section (1 st Time Examinees) Observed LO Section N No. Scores > 75% No. Scores < 75% Threshold (%) Mean Exam Score (%) Chemistry/Immunology 35 33 2 94.3 93.0 Hematology/Urinalysis 35 35 0 100 92.8 Transfusion Medicine 35 29 6 82.9 84.7 Microbiology/Serology 35 35 0 100 93.8 Specimen Collection/Safety 35 35 0 100 83.3 Page 5 of 20
Summary of 2013 2014 Assessment Activities - continued Table 4 ASCP BOC Mean Scaled Scores by Subtest For First Time Examinees Program Mean Scaled Scores: University Programs Mean Scaled Scores BBNK CHEM HEMA IMMU LO MICR UA 568 534 580 558 570 518 583 502 503 509 493 517 502 500 Table 5 ASCP BOC Examination: First Time Examinees Statistics (Annual*) OIT Program Universities National Number of Examinees 34** 2760 1946 Mean Scaled Score 553 503 504 Standard Deviation 90 100 103 Number Passing (Percent) 94.1% 85.3% 84.9 Number Failing (Percent) 5.9% 14.7% 15.1 Minimum Scaled Score Achieved 340 143 135 Maximum Scaled Score Achieved 772 829 876 *Mean of data from July December 2013 cohort of examinees and January June 2014 cohort of examinees STRENGTHS & WEAKNESSES IN STUDENT LEARNING ASCP BOC exam outcomes: Oregon Tech OHSU CLS Program s mean scaled score for first time examinees is 50 points higher than for both University-based programs and National programs mean scaled scores. Additionally, Program mean scaled scores for all sub-content areas are higher than those scores for the University and National Programs. Nevertheless, the minimum acceptable performance for SLO #1was NOT met given that 32 of 34 (94.1%) program first time examinees passed the ASCP BOC with a score greater than or equal to 400. Notably however, one of the 2 students who failed the exam the first time, passed upon subsequent re-testing. ** Students have the right to not release their BOC exam scores to the program. This year, 1 of the 35 students who took the exam chose not to release exam scores. Externship final exam outcomes: two students in the chemistry rotation and six in the transfusion medicine rotation sections did not pass the final exam the first time and required some remediation prior to taking the exam a second time. Following remediation, all students were re-examined and all passed the respective tests with scores above 75%. RECOMMENDATIONS FOR IMPROVEMENTS Faculty made no recommendations for improvements to the program curriculum at this time. Detailed records on assessment activities are kept in the Office of the CLS Department Chair and may be found in individual student records maintained in the CLS program office. Page 6 of 20
Student Learning Outcome #2: Proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data. SLO #2 required curricula listed in the new curriculum map (table B) on page 18. MEASURES OF SLO 1. DIRECT MEASURE: Performance Appraisal at the end of FALL term 2013, faculty assessed professional program students proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data (SLO #2) using externship students performance appraisal scores for problem solving skills: problem recognition (PR)/ problem solving (PS) as determined by clinical site faculty using the professional development evaluation tool. PR is defined as recognizes errors in techniques, results and/or instrumentation malfunctions. PS is defined as shows logical thinking and resourcefulness in dealing with problems/determines course of action to solve problems and/or suggests correct solutions. Students are evaluated by clinical site faculty after completing clinical rotation sections which included: a) chemistry/immunology, b) hematology/urinalysis, c) transfusion medicine, and, d) microbiology/serology. To ensure consistent scoring and to minimize interpretation bias, the program faculty clinical coordinator instructed clinical sites faculty on the use of the tool. 2. DIRECT MEASURE: ISLO Critical Thinking during the WINTER term 2014, entry program students (N = 50) in the CLS 449 Principles of Urinalysis course were assigned a series of case studies. Each student was required to examine patient history, clinical lab data and then make assessments and judgments about the results and interpret the results. To grade each student s work, the course instructor used a rubric that included the ISLO measurement scale. Compiled student scores were submitted to the OIT Assessment Commission the beginning of SPRING term 2014. SLO #2 ASSESSMENT DESIGN & DATA: assessment designs and results of the performance appraisals and the ISLO Critical Thinking exercise are shown in tables 6-8. Table 6 SLO #2 Assessment Design Assessment Performance Criterion Method Proficiency to problemsolve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data. Direct: Performance Appraisals Direct: ISLO Critical Thinking Measurement Scale 3 = Exceeds expectations 2 = Meets expectations 1 = Min. expectations not met N/A = Not observed 1 = Limited or No Proficiency 2 = Developing Proficiency 3 = Proficiency 4 = High Proficiency Min. Acceptable Performance No student receives scores of 1 >95% of students score > 3 in each area: identification, clarification, evaluation Table 7 Externship Students Performance Appraisal Scores: Problem Recognition/Problem Solving Students Professional Evaluation PR/PS Scores by Rotation Section Student Chemistry/Immunology (N=22 of 35*) Hem/UA (N= 23 of 35*) Trans Med (N=22 of 35*) Micro/Serology (N=22 of 35*) PR PS PR PS PR PS PR PS Mean Score 2.6 2.7 2.5 2.6 2.7 2.7 2.6 2.8 *All performance appraisals not available at time of data analysis Page 7 of 20
Summary of Assessment Activities: SLO #2 - continued Table 8 ISLO Critical Thinking Data (N = 50) STRENGTHS & WEAKNESSES IN STUDENT LEARNING No student received a score of 1 for either performance criteria, PR or PS, after completing rotations. Consequently, the minimum acceptable performance for this assessment was met. >95% of students scored > 3 (proficiency) in each area: identification, clarification, evaluation of the Critical Thinking ISLO activity. Consequently, the minimum acceptable performance for this assessment was met. RECOMMENDATIONS FOR IMPROVEMENTS: faculty made no recommendations for changes to the curriculum associated with this SLO. Detailed records on assessment activities are kept in the Office of the CLS Department Chair and may be found in individual student records maintained in the CLS program office. Page 8 of 20
Summary of 2013-2014 Assessment Activities - continued Student Learning Outcome #4: Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions. SLO #4 required curricula listed in the new curriculum map (table C) on page 19. MEASURES OF SLO: 1. DIRECT MEASURE: National Board Exam - at the end of SPRING term 2014 program faculty assessed professional program students administrative skills (SLO #4) using results from the American Society for Clinical Pathology (ASCP) Board of Certification (BOC) Medical Laboratory Scientist (MLS) exam: Laboratory Operations subcontent mean scaled scores. Competency in lab operations is addressed by questions covering the following content areas: Quality Assessment/Troubleshooting, Safety, Management, Laboratory Mathematics, Instrumentation, Molecular Techniques, Education, Communication, and Laboratory Information Systems. 2. INDIRECT MEASURE: Graduate Exit Survey (Student perception of learning outcomes) at the end of FALL term 2013, graduating program student were sent and asked to complete a 26-question exit survey. Question 7 asked students to rate their competency gains at graduation for 13 skills and activities including the ones listed in SLO 4. Faculty assessed professional program students administrative skills (SLO #4) using student responses. SLO #4 ASSESSMENT DESIGN & DATA: assessment designs and results of the ASCP BOC certification exam Lab Operations subcontent and results of the 2013 graduate exit survey question 7 data are shown in tables 9-11. Table 9 SLO #4 Assessment Design Performance Criterion Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions. Assessment Method Direct: National Board Exam Indirect: Graduate Exit Survey Measurement Scale Scaled Mean Score: Lab Operations 1 I exceed the competency expectation for the skill or activity 2 I meet the competency expectation for the skill or activity 3 I meet the minimal expectation for the skill or activity 4 I do not meet the competency expectation for the skill or activity Min. Acceptable Performance >75% of all 1 st time examinees pass Lab Operations subcontent with scores >400 No student responds I do not meet competency expectations... Page 9 of 20
Summary of 2013 2014 Assessment Activities - continued Table 10 ASCP BOC Examination: Lab Operations Subcontent First Time Examinees Statistics (Annual*) OIT Program Universities National Number of Examinees 34 1380 1906 Mean Scaled Score 570* 517* 521* Number Passing (Percent) 28 of 34 82.3% Number Failing (Percent) 6 of 34 (17.6%) Minimum Scaled Score Achieved 270 Maximum Scaled Score Achieved 999 Data Not Available *Mean of data from July December 2013 cohort of examinees and January June 2014 cohort of examinees Table 11 2013 Graduate Exit Survey Question 7 (N =23 of 35): Competency Gains & Skills Skill or activity 1 - I exceed the competency expectation for the skill or activity Scale 2 I meet the competency expectation for the skill or activity 3 I meet the minimal expectation for the skill or activity Number of Responses 4 I do not meet the competency expectation for the skill or activity Quality Assurance/CQI 9 12 2 0 Laboratory Education 8 12 3 0 Management 4 13 6 0 Appropriate composure 20 3 0 0 *Mean of data from July December 2013 cohort of examinees and January June 2014 cohort of examine STRENGTHS & WEAKNESSES IN STUDENT LEARNING Twenty-eight of thirty-four or 82.3% first time examinees pass the Lab Operations subcontent section of the ASCP BOC exam. Consequently, the minimum acceptable performance for this assessment was met. Likewise, none of the 23 of 35 graduates who completed the exit survey responded to question 7 I do not meet competency expectations... RECOMMENDATIONS FOR IMPROVEMENTS: faculty made no recommendations for changes to the curriculum associated with this SLO. Detailed records on assessment activities are kept in the Office of the CLS Department Chair and may be found in individual student records maintained in the CLS program office. Page 10 of 20
Student Learning Outcome #6: Effective communication skill to ensure accurate and appropriate information transfer. SLO #6 required curricula listed in the new curriculum map (table D) on page 20. 1. MEASURE OF SLO 1. DIRECT MEASURE: Performance Appraisal at the end of FALL term 2013, faculty assessed students effective communication skills (SLO #6) using externship students performance appraisal scores for communication as determined by clinical site faculty using the professional development evaluation tool. Communication attributes assessed are defined as: LW - ability to listen well and RG - ability to receive/give information to others effectively and courteously. Students are evaluated by clinical site faculty after completing clinical rotation sections which included: a) chemistry/immunology, b) hematology/urinalysis, c) transfusion medicine, and, d) microbiology/serology. To ensure consistent scoring and to minimize interpretation bias, the program faculty clinical coordinator instructed clinical sites faculty on the use of the tool. SLO #6 ASSESSMENT DESIGN & DATA: assessment design and results of the performance appraisal are shown in tables 12 & 13. Table 12 SLO #6 Assessment Design Assessment Performance Criterion Method Measurement Scale Min. Acceptable Performance Effective communication skill to ensure accurate and appropriate information transfer Direct: Performance Appraisals 3 = Exceeds expectations 2 = Meets expectations 1 = Min. expectations not met N/A = Not observed No student receives scores of 1 Table 13 Externship Students Performance Appraisal Scores: Effective Communication Students Professional Evaluation PR/PS Scores by Rotation Section Student Chem/Immunology (N=22 of 35*) Hem/UA (N= 23 of 35*) Trans Med (N=20 of 35*) Micro/Serology (N=22 of 35*) LW RG LW RG LW RG LW RG Mean Score 2.8 2.9 2.8 2.7 2.8 2.8 2.9 2.9 *All performance appraisals not available at time of data analysis STRENGTHS & WEAKNESSES IN STUDENT LEARNING No student received a score of 1 for either performance criteria, LW or RG, after completing rotations. Consequently, the minimum acceptable performance for this assessment was met. RECOMMENDATIONS FOR IMPROVEMENTS: faculty made no recommendations for changes to the curriculum associated with this SLO. Detailed records on assessment activities are kept in the Office of the CLS Department Chair and may be found in individual student records maintained in the CLS program office. Page 11 of 20
V. Evidence of Student Learning SLO # 1 - Competency to perform a full range of testing in the contemporary medical laboratory encompassing pre-analytical, analytical, and post-analytical components of laboratory services a. Strengths in student learning: 32 of 34 first time examines passed the national MLS certification exam as evidenced by a program scaled mean score of 553. This score is well above the minimum pass score of 400 and 50 points above the mean scaled scores for Universitybased programs (222) and the National mean scaled score (1946 examinees). Additionally, 35 of 35 first time examinees passed (scores > 75%) the final exam administered in 3 of the 5 externships (Hematology/Urinalysis, Microbiology/Serology, and Specimen Collection/Safety). The mean final exam score (%) for the 35 students completing each of the five externships was 8-19% above the minimal pass score of 75%. b. Weaknesses in student learning faculty noted that the students who did not pass the national MLS certification exam were the same two students whose academic ranking (cumulative and/or term GPAs) was consistently in the lowest 25% of their class of 35. Additionally, the majority of the first time examinees (6 of 8) who did not pass the transfusion medicine or the chemistry externship exam held academic rank in the lowest 25% of their class based on term (Fall 2013) and cumulative GPA. c. Program improvements see comments in section VI. SLO #2 - Proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data a. Strengths in student learning minimal expectations for this SLO were met or exceeded, notably the same reported finding for the previous two years year (2011 and 2012). b. Weaknesses in student learning - none noted at this time for this SLO c. Program improvements none at this time SLO #4 - Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions. a. Strengths in student learning Data indicates that minimal acceptable performance was met or exceeded for this SLO. Notably, this finding represents no change from the previously reported outcome for this SLO the last two most recent times it was assessed (2008-2009 and 2011-2012). b. Weaknesses in student learning student learning gains were assessed using one direct and one indirect measure. In each case, the failures of a few students to meet benchmarks for the performance criteria was not interpreted as weaknesses in student learning requiring curriculum changes. c. Program improvements none at this time SLO #6 - Effective communication skill to ensure accurate and appropriate information transfer. a. Strengths The last time this SLO was assessed was 2009. As was the case then, all students assessed this year met or, in most cases, exceeded minimal performance expectations. b. Weaknesses in student learning - none noted at this time c. Program improvements none at this time Page 12 of 20
VI. Clinical Laboratory Science (CLS) Program Assessment Report Changes Resulting from Assessment a. Changes: As a result of the continued positive feedback regarding the use of a mock board exam administered on-line through Media Lab, Inc. the program will continue to make it available to students. (See VI a. on page 14 of the 2012 assessment report for historical context information). b. Re-assessment: Review book feedback from the class of 2013 was positive. Faculty recommendation of review books and inclusion of review books in the program s textbook list will continue. (See VI b. on page 14 for historical context information). a. Results from prior assessment compared to re-assessment: see table 14 on page 14. b. Summary of Findings and Changes for permanent program changes or new actions: 1. Beginning with this report, both indirect and direct measures of SLO were reported and analyzed (SLO 4). (See VI d3. on page 14 of the 2012 assessment report for historical context information). 2. Though the minimal performance expectation for SLO 1, direct measure 2: certification exam scores, was not met, faculty did not recommend making changes to curriculum for the following reasons: a. The ASCP BOC subtest scaled mean scores for 2013 examinees was not statistically different from the subcontent mean scores for examinees of the previous four years [F(4, 30) = 2.31, p = 0.081]. b. The Oregon Tech OHSU CLS program is accredited through the National Accrediting Agency for Clinical Laboratory Sciences (NAACLS). In 2012, NAACLS adopted new standards for all its accredited and approved programs. For the Oregon Tech CLS program, these new standards take effect in 2014. The new Assessment and Continuous Quality Improvement standard IIB requires: A review of the results of the following outcomes measures (includes external certification results) from at least the last three active years must be documented, analyzed and used in program assessment and continuous quality improvement of the program to include an annual submission to NAACLS. If outcome measure(s) does/do not meet the stated NAACLS approved benchmarks, then an analysis and action plan must be submitted to correct the deficiency (ies). The benchmark for the ASCP BOC exam for all MLS accredited programs is Three years consecutive results of graduate certification rates demonstrating an average of at least 75% pass rate on the BOC examinations, for those who take the exam within the first year of graduation as calculated by the most recent three year period. Include source documentation with student names redacted. While the Oregon Tech OHSU CLS program will continue to use and report ASCP BOC examination performance as one of its SLO s, it will do so beginning 2014 under a new assessment plan designed to align with the newest NAACLS accreditation standards. This change includes adopting the outcome measures and adhering to the approved NAACLS benchmarks specified in NAACLS standard IIB. Page 13 of 20
Table 14 ASCP BOC Total and Subtest Mean Scaled Scores for First Time Examinees 2009-2013 2009 OIT OHSU CLS Program s Mean Scaled Scores (N=24) University Program s Mean Scaled Scores (N=2424) Total Score BBNK CHEM HEMA IMMU LO MICR UA 560 602 530 575 546 584 533 572 472 494 483 494 483 508 485 481 2010 OIT OHSU CLS Program s Mean Scaled Scores (N=22) University Program s Mean Scaled Scores (N=2276) Total Score BBNK CHEM HEMA IMMU LO MICR UA 536 582 481 574 583 542 505 526 492 495 492 496 488 518 487 485 2011 OIT OHSU CLS Program s Mean Scaled Scores (N=35) University Program s Mean Scaled Scores (N=2251) Total Score BBNK CHEM HEMA IMMU LO MICR UA 581 599 561 599 560 598 560 617 502 500 501 512 494 519 499 495 2012 OIT OHSU CLS Program s Mean Scaled Scores (N=26) University Program s Mean Scaled Scores (N=2539) Total Score BBNK CHEM HEMA IMMU LO MICR UA 580 591 558 615 561 656 542 576 501 502 498 510 490 526 500 491 2013 OIT OHSU CLS Program s Mean Scaled Scores (N=34) University Program s Mean Scaled Scores (N=2539) Total Score BBNK CHEM HEMA IMMU LO MICR UA 553 568 534 580 558 570 518 593 503 502 503 509 493 517 502 500 ANOVA statistical test performed Source of Sum of d.f. Mean F Variation Squares Squares between 9504. 4 2376. 2.312 (Fcritical ~ 2.416) error 3.0833E+04 30 1028. total 4.0338E+04 34 The probability of this result, assuming the null hypothesis no difference in means, is 0.081 Page 14 of 20
VII. References Clinical Laboratory Science (CLS) Program Assessment Report a. Calvo, C. Oregon OHSU Clinical Laboratory Science Program Annual Assessment Report 2012 2013. Available at URL http://www.oit.edu/faculty-staff/provost/learning-outcomes/clinicallaboratory-science b. Furman, A. Oregon OHSU Clinical Laboratory Science Program Annual Assessment Report 2011 2012; also reports for 2009 2010 and 2010-2011. Available at URL http://www.oit.edu/facultystaff/provost/learning-outcomes/clinical-laboratory-science c. Wolfe, T. 2013 2014 Oregon Tech ISLO: Critical Thinking Data (winter term 2014). Submitted electronically on 4/29/2012 to the Oregon Tech Assessment Commission by C. Calvo. d. National Accrediting Agency for Clinical Laboratory Sciences (NAACLS). 2012 Guide to Accreditation and Approval. Chicago, Illinois, 2013. Accessible from URL http://naacls.org/accreditation/mls/ e. National Accrediting Agency for Clinical Laboratory Sciences (NAACLS). 2012 Standards Compliance Guide. Chicago, Illinois, 2013. Accessible from URL http://naacls.org/accreditation/mls/ Page 15 of 20
VIII. Clinical Laboratory Science (CLS) Program Assessment Report Appendix CLS Program Curriculum Maps for Student Learning Outcomes Page 17 - Table A Curriculum Map for SLO 1 Page 18 - Table B Curriculum Map for SLO 2 Page 19 - Table C Curriculum Map for SLO 4 Page 20 - Table D Curriculum Map for SLO 6 Page 16 of 20
SLO #1: Competency to perform a full range of testing in the contemporary medical laboratory encompassing pre-analytical, analytical, and post-analytical components of laboratory services Table A - Curriculum MAP for SLO 1; required courses are italicized CLS 420 Clinical Immunology 4 FALL CLS 432 Foundations of CLS I 4 CLS 424 Hemostasis 3 CLS 444 Microbiology I 6 CLS 415 Clinical Chemistry I 6 WINTER CLS 447 Practicum: Chemistry 6 CLS 445 Microbiology II 4 CLS 449 Principles of Urinalysis 2 CLS 416 Clinical Chemistry II 2 SPRING CLS 442 Hematology I 6 CLS 443 Immunohematology I 4 CLS 446 Microbiology III 4 CLS 422 Molecular Diagnostic Methods 2 CLS 448 Infectious Serology 1 SUMMER CLS 452 Hematology II 5 CLS 453 Immunohematology II 2 CLS 457 Adv. Chem/Immunology Concepts 2 CLS 462 Foundations of CLS II 2 CLS 470 Chemistry and Immunology Externship 4 FALL CLS 471 Hematology Externship 4 (2 nd year) CLS 472 Microbiology Externship 4 CLS 473 Immunohematology Externship 4 Page 17 of 20
SLO #2: Proficiency to problem-solve, troubleshoot, and interpret results, and to use statistical approaches when evaluating data. Table B - Curriculum MAP for SLO 2; required courses are italicized CLS 420 Clinical Immunology 4 FALL CLS 432 Foundations of CLS I 4 CLS 424 Hemostasis 3 CLS 444 Microbiology I 6 CLS 415 Clinical Chemistry I 6 WINTER CLS 447 Practicum: Chemistry 6 CLS 445 Microbiology II 4 CLS 449 Principles of Urinalysis 2 CLS 416 Clinical Chemistry II 2 SPRING CLS 442 Hematology I 6 CLS 443 Immunohematology I 4 CLS 446 Microbiology III 4 CLS 422 Molecular Diagnostic Methods 2 CLS 448 Infectious Serology 1 SUMMER CLS 452 Hematology II 5 CLS 453 Immunohematology II 2 CLS 457 Adv. Chem/Immunology Concepts 2 CLS 462 Foundations of CLS II 2 CLS 470 Chemistry and Immunology Externship 4 FALL CLS 471 Hematology Externship 4 (2 nd year) CLS 472 Microbiology Externship 4 CLS 473 Immunohematology Externship 4 Page 18 of 20
SLO #4: Administrative skills consistent with philosophies of quality assurance, continuous quality improvement, laboratory education, fiscal resource management, and appropriate composure under stressful conditions. Table C - Curriculum MAP for SLO 4; required courses are italicized CLS 420 Clinical Immunology 4 FALL CLS 432 Foundations of CLS I 4 CLS 424 Hemostasis 3 CLS 444 Microbiology I 6 CLS 415 Clinical Chemistry I 6 WINTER CLS 447 Practicum: Chemistry 6 CLS 445 Microbiology II 4 CLS 449 Principles of Urinalysis 2 CLS 416 Clinical Chemistry II 2 SPRING CLS 442 Hematology I 6 CLS 443 Immunohematology I 4 CLS 446 Microbiology III 4 CLS 422 Molecular Diagnostic Methods 2 CLS 448 Infectious Serology 1 SUMMER CLS 452 Hematology II 5 CLS 453 Immunohematology II 2 CLS 457 Adv. Chem/Immunology Concepts 2 CLS 462 Foundations of CLS II 2 CLS 470 Chemistry & Immunology Externship 4 FALL CLS 471 Hematology Externship 4 (2 nd year) CLS 472 Microbiology Externship 4 CLS 473 Immunohematology Externship 4 Page 19 of 20
SLO #6: Effective communication skill to ensure accurate and appropriate information transfer. Table D - Curriculum MAP for SLO 6; required courses are italicized CLS 420 Clinical Immunology 4 FALL CLS 432 Foundations of CLS I 4 CLS 424 Hemostasis 3 CLS 444 Microbiology I 6 CLS 415 Clinical Chemistry I 6 WINTER CLS 447 Practicum: Chemistry 6 CLS 445 Microbiology II 4 CLS 449 Principles of Urinalysis 2 CLS 416 Clinical Chemistry II 2 SPRING CLS 442 Hematology I 6 CLS 443 Immunohematology I 4 CLS 446 Microbiology III 4 CLS 422 Molecular Diagnostic Methods 2 CLS 448 Infectious Serology 1 SUMMER CLS 452 Hematology II 5 CLS 453 Immunohematology II 2 CLS 457 Adv. Chem/Immunology Concepts 2 CLS 462 Foundations of CLS II 2 CLS 470 Chemistry & Immunology Externship 4 FALL CLS 471 Hematology Externship 4 (2 nd year) CLS 472 Microbiology Externship 4 CLS 473 Immunohematology Externship 4 Page 20 of 20