Physical Therapist Assistant Assessment Report Program / Discipline Assessment Report Program/Discipline: Physical Therapist Assistant Program Responsibility: Paula Provence Program/Discipline's Mission Statement: The Physical Therapist Assistant (PTA) program provides physical therapist assistant education to a widely diverse student clientele. Since the educational needs of the program's constituency vary significantly, the program is committed to maintaining instructional flexibility to provide for these diverse needs; it encourages lifelong learning. The PTA program is also committed to improving student learning through continual assessment of student achievement. The Physical Therapist Assistant program fosters a sense of personal and professional integrity, ethics and responsibility, and understanding of the critical thinking and problem solving processes; it develops communication skills and promotes an awareness and acceptance of individuals and groups from various backgrounds. The physical therapist assistant program's commitment to this educational process will prepare students with the medical knowledge, clinical skills, and professional behaviors necessary to meet the challenges inherent within our complex, evolving systems of physical therapy and health care delivery. The Physical Therapist Assistant program responds to our physical therapy community including efforts to advance the recognition of the field of physical therapy and the role of PTAs within the field. We partner with area clinicians and facilities to determine and meet their needs for physical therapist assistants. Purpose: The purpose of the PTA program is to produce graduate PTAs prepared to competently perform the duties of a PTA in a variety of PT settings and to successfully complete the national license examination and obtain licensure in the state of their choice. Program/Discipline's Assessment History: By using the assessment process as an evaluative technique, how has it previously affected your program's curricula and/or teaching strategies? The PTA program has been assessing the ability of graduates to function at an entry level in a variety of setting for several years. Now that our curriculum has stabilized from the mandated changes of Common Numbering we can again look at a two cycle result period to consider cause and effect between curriculum and results. PTA students continue to perform well in areas assessed and become successfully employed in the community. Despite positive results the program continues to consider improvements to our program. Changes in the PTA program curricula are more reflective of retention rates than assessment performance. As previous assessments have shown, program goals are being well met by the students who complete the program. However, retention rates have resulted in our re-sequencing the curriculum for increased retention.
By using the assessment process as an evaluative technique, what changes to student learning have been noted? The PTA program assessment has confirmed that our graduates are obtaining the level of education and training required for program objectives to be met. What unintended consequences, if any, have occurred because of the assessment process? We have not experienced any unintended consequences. Who receives information about your department's assessment and why? (Please note if you plan on altering either of these items for the coming year.) Program assessment information is shared with students, faculty, our advisory committee and the PT community at large. It is also communicated to the Commission on Accreditation of Physical Therapy Education albeit in a different format. Part 1: Previous Academic Year Assessment Summary Previous Academic Year: 2007-2008 Outcome #: 1 Outcome Type (choose by bolding): Discipline/Program; Other Benchmark for success 1) Please specify what percentage of the sample size is expected to meet or exceed your benchmark. 2) What is the rationale for choosing this measure? Outcome Title: Entry Level Clinical Skills Outcome Description: "Demonstrate entry level clinical skills required to perform the duties if a Physical Therapist Assistant in a variety of practice settings." Method #1 Benchmark: Students in PTA 282 will achieve entry level competency in at least 13 of the 16 clinical skills included in the PTA Clinical Performance Instrument. Entry level competence is at the 100% mark on the scale of 0-100%. Method #2 Benchmark: 95% of the employers will rate the students clinical skill performance as acceptable or higher. Method #1 Rationale: Our benchmark was set at 13 of 16 clinical skills because there are a few skills in the Clinical Performance Instrument which cannot be completed in every facility. For example, skill #18 involves demonstrating the ability to appropriately delegate components of interventions to physical therapy aides and other support personnel. Not every clinic utilizes support personnel; therefore, it is appropriate to allow for this exception in the benchmark rather than need to exclude the student from the results based on such an occurrence. Method #2 Rationale: Our benchmark on this indicator has been set at 95% to acknowledge individual characteristics of students and employers. The inherent small sample size created by the number of students in this program could otherwise create a skewed result. Description of assessment process: 1) What assessment methods were used to measure this outcome (i.e. pre/post test, portfolio review, etc.)? Method #1: Students' clinical skills will be assessed in their final clinical internships (PTA 282) by clinical instructors using the PTA Clinical Performance Instrument (version 98).
2) How do these methods show students are learning? 3) What frequency is this outcome being measured (i.e.: each semester, yearly, every other year, etc.) and why? 4) How many students made up the sample size? The 16 clinical skills measured by this assessment include: 1. Performs in a safe manner that minimizes risk to patient, self, and others. 6. Communicates in ways that are congruent with situational needs. 7. 8. 9. 10. 11. 12. 13. Produces documentation to support the delivery of physical therapy services Delivers established patient care to reflect respect for and sensitivity to individual differences. Participates in patient status judgments within the clinical environment based on the plan of care established by the physical therapist Obtains accurate information by performing selected data collection consistent with the plan of care established by the physical therapist. Discusses the need for modifications to the plan of care by the physical therapist. Performs physical therapy interventions in a technically competent manner. Educates others (patients, family, caregivers, staff, students, other health care providers) using relevant and effective teaching methods. 14. Participates in activities addressing quality of service delivery. 15. 16. Participates in addressing patient needs for services other than physical therapy. Manages resources (e.g. time space and equipment) to achieve goals of the clinical setting. 17. Participates in fiscal management of the physical therapy setting. 18. 19. 20. Uses physical therapy aides and other support personnel according to legal standards and ethical guidelines. Implements a self directed plan for career development and lifelong leaning. Assists the physical therapist in addressing primary and secondary prevention needs of individuals and groups. While the evaluation tool is completed by students and their clinical instructors as a formative
evaluation earlier in their educational program, its use during PTA 282 is a summative evaluation. At the completion of this clinical affiliation, students have completed all didactic and clinical coursework. This year our sample size is 13. Method #2 Students clinical skills will be assessed through employer surveys. Results What were the results of the assessment process? (List results for each method, if more than one were used.) Results Method #1 Student Total Competencies Completed Competencies completed at 100% Competency number and competency level if less than 100% Competency not observed during internship 1 16 16 N/A N/A 2 16 16 N/A N/A 3 16 12 10, 11, 19 95% 14 4 16 16 N/A N/A 5 16 16 N/A N/A 6 16 16 N/A N/A 7 16 16 N/A N/A 8 16 16 N/A N/A 9 16 16 N/A N/A 10 16 16 N/A N/A 11 16 16 N/A N/A 12 16 15 N/A 13 13 16 15 N/A 16
Method #2 Results: Employer survey data was not available at the time this report was submitted. If it becomes available in the near future, an amended report will be submitted. What did the department learn? 1) How did group performance compare to the benchmark? 2) How does the data compare to the previous year, if applicable? 3) If multiple measures were used, how do they compare to each other? 100% of the 13 students met the benchmark of achievement of entry level competency in at least 13 of 16 clinical skills included in the PTA Clinical Performance Instrument. Competency Number 14 may not be addressed at all facilities based on type of facility and services provided. The omission of these competencies does not cause concern. Competency Number 14 focuses on addressing student participation in activities addressing quality of service delivery. It is very possible that department quality improvement activities do not correspond with the 6 weeks our student is in the clinic. Competencies 13 and 16 are skills not usually unobserved during a clinical affiliation. The omission of these competencies have been flagged for follow up with each facility. Competency number 13 focuses on education and competency 16 deals with management of resources. In comparing the results of this assessment cycle with the previous one an increase is noted in skills observed while the percentage of skills not reaching the 100% competency decreased and our benchmark for this outcome was met. Student performance summary 1) Based on the findings, how does the department rate student performance in regards to this outcome (strong, weak, or neutral)? 2) How does this assessment affect plans for this coming year in terms of curricula, teaching strategies, and assessment methods? The results achieved by these indicators allows the department to continue to rate student performance and our achievement of program outcomes as excellent. Despite our track record of achieving these results the program will continuously monitor this outcome to assure that changes in the profession, our student and applicant pool or the external environment do not result in any less.
Outcome #: 2 Outcome Type (choose by bolding): Discipline/Program; Other Benchmark for success 1) Please specify what percentage of the sample size is expected to meet or exceed your benchmark. 2) What is the rationale for choosing this measure? Outcome Title: Professional Behavior Outcome Description: Demonstrate the professional behaviors required to perform the duties of a Physical Therapist Assistant in a variety of practice settings. Method #1 Benchmark: Students in PTA 282 will achieve entry level competency in at least 6 of the 7 professional behavior skills included in the PTA Clinical Performance Instrument. Entry level competence is at the 100% mark on the scale of 0-100%. Method #2 Benchmark: 95% of the employers will rate the students professional behaviors as acceptable or higher. Method #1 Rationale: Our benchmark was set at 7 of 7 clinical skills because there these skills should be attainable completed in every facility. Method #2 Rationale: Our benchmark on this indicator has been set at 95% to acknowledge individual characteristics of students and employers. The inherent small sample size created by the number of students in this program could otherwise create a skewed result. Description of assessment process: 1) What assessment methods were used to measure this outcome (i.e. pre/post test, portfolio review, etc.)? 2) How do these methods show students are learning? 3) What frequency is this outcome being measured (i.e.: each semester, yearly, every other year, etc.) and why? 4) How many students made up the sample size? Method #1: Students' professional behaviors were assessed in their final clinical internships (PTA 282) by clinical instructors using the PTA Clinical Performance Instrument (version 98). The 7 professional behavior skills measured by this assessment included: 2. Conducts self in a responsible manner. 3. Interacts with others in a respectful manner. 4. Adheres to ethical standards. 5. Adheres to legal standards. 6. Communicates in ways that are congruent with situational needs. 8. 19. Delivers established patient care to reflect respect for and sensitivity to individual differences. Implements a self directed plan for career development and lifelong leaning.
Method #2: Students professional behaviors will be assessed through employer surveys. While the evaluation tool is completed by students and their clinical instructors as a formative evaluation earlier in their educational program, its use during PTA 282 is a summative evaluation. At the completion of this clinical affiliation, students have completed all didactic and clinical coursework. This year our sample size is 13. Results What were the results of the assessment process? (List results for each method, if more than one were used.) Results Student Total Competencies Completed Competencies completed at 100% Competency number and competency level if less than 100% Competency not observed during internship 1 7 7 N/A N/A 2 7 7 N/A N/A 3 7 6 19 95% N/A 4 7 7 N/A N/A 5 7 7 N/A N/A 6 7 7 N/A N/A 7 7 7 N/A N/A 8 7 7 N/A N/A 9 7 7 N/A N/A 10 7 7 N/A N/A 11 7 7 N/A N/A 12 7 7 N/A N/A 13 7 7 N/A N/A Results Method #2: Employer survey data was not available at the time this report was submitted. If it becomes available in the near future, an amended report will be submitted. What did the department learn? 1) How did group performance compare to the benchmark? 2) How does the data compare to the previous year, if applicable? 92% of the 13 students met the benchmark of achieving entry level competence on 7 of the 7 identified skills. The remaining 8% (one student) completed this skill at the 95% level. This data is comparable to the results of the previous assessment cycle in that all of the students achieved 100% in the skills that were rated. Our small sample size creates exaggerated percentage
3) If multiple measures were used, how do they compare to each other? changes for each student. Consequently, data review is required for appropriate perspective. The identification of professional behavior skills not observed did not occur this cycle or the previous cycle fulfilling our 2 year trend requirement. Student performance summary 1) Based on the findings, how does the department rate student performance in regards to this outcome (strong, weak, or neutral)? 2) How does this assessment affect plans for this coming year in terms of curricula, teaching strategies, and assessment methods? The results achieved by these indicators allows the department to continue to rate student performance and our achievement of program outcomes as excellent. Despite our track record of achieving these results, the program will continuously monitor this outcome to assure that changes in the profession, our student and applicant pool or the external environment do not result in any less.
Outcome #: 3 Outcome Type (choose by bolding): Discipline/Program; General Ed; If General Education outcome (choose by bolding): Communication; Critical Thinking; Quantitative Reasoning; Use of Technology; Diversity and Global Awareness; Leadership and Teamwork Benchmark for success 1) Please specify what percentage of the sample size is expected to meet or exceed your benchmark. 2) What is the rationale for choosing this measure? Outcome Title: Math/algebra Skills Outcome Description: Demonstrate math/algebra skills required for application of principles learned in Kinesiology and Modalities in PT. Benchmark: 100% of the PTA students will have achieved Accuplacer Algebra scores of at least a 45. Benchmark: 90% of the PTA students tested in PTA120 and PTA 140 will receive no less than 85% of the points possible on these exam questions. Description of assessment process: 1) What assessment methods were used to measure this outcome (i.e. pre/post test, portfolio review, etc.)? 2) How do these methods show students are learning? 3) What frequency is this outcome being measured (i.e.: each semester, yearly, every other year, etc.) and why? 4) How many students made up the sample size? Results What were the results of the assessment process? (List results for each method, if more than one were used.) Method #1: Students math/algebra skills will be assessed at program admission using Accuplacer scores. Method #2: Students math/algebra skills will be assessed through examination questions in PTA 120 and PTA 140 which require use of algebra skills to correctly administer PT modalities and exercise concepts. Method #1 Student Admit CPT- Algebra 1 50 2 62 3 75 4 71 5 118 6 63 7 70 8 87 9 75 10 88 11 75 12 75 13 105
Method #2 Question completed at 100% Question Received Partial Credit Points lost secondary to Math Skills Points lost secondary to Course Concept PTA 140 58% 33% 0% 8% PTA 120 35% 62% 15% 47% What did the department learn? 1) How did group performance compare to the benchmark? 2) How does the data compare to the previous year, if applicable? 3) If multiple measures were used, how do they compare to each other? Student performance summary 1) Based on the findings, how does the department rate student performance in regards to this outcome (strong, weak, or neutral)? 2) How does this assessment affect plans for this coming year in terms of curricula, teaching strategies, and assessment methods? Statistically student performance on this indicator did not meet our benchmark. However, with further investigation we can extrapolate the data to show compliance with this minimal standard. Ultimately, the structure of the outcome, benchmark, data and results did not produce the useful results we had hoped. Small sample size and too many variables left us with no real conclusion. We will restructure our efforts for the next assessment cycle.
Part 2: Current Academic Year Assessment Plan Introduction to a more flexible approach Assessment planning is more flexible than in previous years. Your program may wish to examine how retention differs among sections of an important course or you may choose to do an analysis of grade inflation across courses within your program/discipline. Such items aren't truly student outcomes, but they certainly affect learning. As such, these outcomes will be classified as "Other" in the summary you create next year. (Measuring such outcomes is purely optional.) Two or more instruments of measuring an objective may provide greater clarity and validity, but only one is required. The department or program makes the decision. The Program Assessment committee and deans are available for consultation. In the past, some programs have been identified purely by prefix or in some cases by the type of section offered. Sometimes, a very limited pool of students have been available for such a program to assess, or the program lacks full-time faculty to plan, assess, and report outcomes. If your program has such difficulties, please contact either the Program Assessment committee's chair or your School's Program Assessment committee representative. We will work with you to find a solution. CTE programs with external accreditation may use the accreditation report to in addition or in lieu of these forms, please contact the Program Assessment committee representative if this format is desired. In absence of this contact, these forms are expected. Outcomes are to be measured annually. Exceptions are made with VPI approval for outcomes that clearly need a less (or more) frequent review. Outcome minimums At least two outcomes are to be program-related. At least two outcomes are to be General Education in nature. One General Education outcome must be continued from the prior year to develop a historical trend. General Education outcomes need to be assessed and reported annually, regardless of the frequency of reporting for other outcomes. Both outcomes above are classified as "student learning" outcomes, requiring benchmarks and analysis. It is strongly recommended that you use the table provided in Part 1 of this report for this function. Definitions and examples of these outcomes are provided in Appendix A at the end of this document. Your Program Assessment committee is available to assist. An assessment report is requested annually. Such a report may only consist of a report on General Education outcomes and a plan summarizing where your program is in an assessment with multi-year frequency.
Current Academic Year: 2008-2009 Intended Learning Outcomes (only include if they differ from those noted in Part 1) The PTA program will continue to monitor achievement of clinical and behavior skill attainment in our final student affiliation. In addition we will add the following outcome: Program admission requirements accurately identify the level of writing and math skills required to support mastery of Physical Therapy content leading to an overall program retention rate of 75%. Assessment Method(s) (only include if they differ from those noted in Part 1) Because of continuing difficulties with acquiring employer survey data which provides meaningful results, the PTA program will suspend use of this method for our first two outcomes until we can establish a program based employer survey. For our new Outcome #3 Method #1 Compare Accuplacer reading comprehension scores of each student with final grades of PTA 110, HPR 117, PTA 120, PTA 140 for positive correlation. Method #2 Compare Accuplacer algebra scores of each student with final grades of PTA 110, HPR 117, PTA 120, PTA 140 for positive correlation. Benchmarks (only include if they differ from those noted in Part 1) A correlation value of r >=.5 will exist for each method. Have you submitted a separate budget worksheet? (Choose by bolding; for information about this worksheet, please refer to the specific budgeting e-mail sent by the committee chairperson.) Yes No Please submit this report (including both last year's summary and this year's plan) in a Word document to the Program Assessment committee chairperson (Tom DeMoulin: tom.demoulin@arapahoe.edu). If you have any questions about the process, please contact the chairperson.
ADDENDUM A: EXAMPLES OF LEARNING OUTCOMES Outcomes should: (a) be stated in terms of expected student behaviors/achievements; (b) be measurable; (c) be aggregate, focusing on the discipline and not on individual students or course sections: (d) specify the skills, competencies, understandings, and values that students should have acquired as result of having completed courses within the discipline; (e) focus on the learning resulting from an activity rather than on the activity itself; (f) aligned with course, academic program, and institutional levels; (g) focus on aspects of learning that will endure and can be assessed GOOD EXAMPLES OF LEARNING OUTCOMES: Students will be able to demonstrate the ability to produce basic drawing projects to scale. Students will be able to apply standard conventions of grammar, usage, and mechanics in a written essay. Academy students will be required to perform a building search exercise demonstrating the appropriate application of the use of force to include weapon control. BECAUSE: Concise; achievement oriented; measurable, & specific. Triple-barreled, but combination of skills important in essay writing. Measurable, specific, and stated in terms of an achievement. Specific; aggregate in scope; measurable, & stated in terms of an achievement. POOR EXAMPLES OF LEARNING OUTCOMES: Summarize the key concepts outlined in the course. Students will show an appreciation for the theories and practice of the field. Students will pass national licensure examination. BECAUSE: Summarize does not indicate level of learning expected. Demonstrate, Apply, Contrast, Integrate and/or Synthesize would be better. Seems to focus on just one course. Difficult to understand what the students will do to demonstrate this ability. Vague and general. This outcome could be improved by stating as Students will show an appreciation for the theories and practices of the field by demonstrating broader application to real-life situation, career interests, other areas of study Passing alone does not provide information about specific strengths or weaknesses in skills or competency levels. Definitions of General Education outcomes: 1) Communication: Students will demonstrate a college-level ability to read and to communicate effectively through speaking, writing, listening, and artistic expression. 2) Critical Thinking: Students will develop the skills necessary to understand and apply mathematical concepts and reasoning and to analyze and interpret various types of data. 3) Quantitative Reasoning: Students will demonstrate the ability to analyze, synthesize, evaluate, make decisions, think critically and creatively, solve problems effectively; and apply and extend knowledge to new environments and situations. 4) Use of Technology: Students will use appropriate and current technologies to collect, retrieve, organize, and process information from various sources and produce original work in various ways. 5) Diversity and Global Awareness: Students will demonstrate knowledge, appreciation, and understanding of individual and social responsibility, good citizenship, and diverse cultural customs, beliefs, traditions, and lifestyles. 6) Leadership and Teamwork: Students will develop effective leadership, teamwork, relationship management, and conflict resolution skills.
ADDENDUM B: EXAMPLES OF ASSESSMENT METHODS GOOD EXAMPLES OF SINGLE ASSESSMENT METHODS: Pre Test / Post Test administered at the beginning and end of a unit, term, year, or degree. Portfolio of student work BECAUSE: Direct method, which isolates how much a student knows before starting the unit. Are your students doing well because your program is teaching them well or because they are entering your program with much of the knowledge or skills already? Provides a longitudinal study of how the student has progressed through a program. Such a portfolio can be designed to address specific outcomes, addressing the "general" problem to the right. POOR EXAMPLES OF SINGLE ASSESSMENT METHODS: Student and employer surveys Grades ADDENDUM C: EXAMPLES OF BENCHMARKS Benchmarks should: (a) be specific, concise (b) be linked to each learning outcome (c) be realistic, attainable (d) indicate the number of students expected to meet the criteria GOOD EXAMPLES OF BENCHMARKS: 80% of students will score Good Quality or better on all of the five elements of basic drawing projects. Student essays at end-of-course will be significantly better (at p<.05 level) in terms of grammar, usage, and mechanics than beginning-ofcourse essays as measured by a comparison of paired essays. Out of three exercises, all graduating Academy students will be able to perform at least one building search exercise without major errors and without more than one minor error. BECAUSE: Specific; refers to level and elements of scoring rubric; benchmark directly linked to Learning Outcome. Refers to specific procedure and assessment methodology; benchmark directly linked to Learning Outcome. Specific, realistic, attainable; benchmark directly linked to Learning Outcome. POOR EXAMPLES OF BENCHMARKS: Students will achieve an overall level of proficiency in course concepts. Student responses were rated on a scale of 1-5. Students will surpass performance levels attained during previous assessment cycle. BECAUSE: Indirect methods, relying on perception rather than quantifiable data. However, such a method is often valuable as a second measurement of an outcome when combined with a direct method. For instance, a student believing he has learned more than demonstrated poses a different challenge to a department than a student who knows she hasn't learned a concept. Unless a grade can be specifically isolated to a particular outcome, it is usually too general. BECAUSE: Does not indicate how many students; level of overall proficiency too vague. Does not indicate what level of performance the group is expected to attain. Does not provide information about how many students will perform better, or by how much.