Annual Progress Report on Assessment of Student Learning for Undergraduate Programs (Rev 04/11) The purpose of the Annual Progress Report (APR) of Student Learning Assessment is to provide a continuous improvement process through meaningful assessment of student. Results from assessment of student guide collective actions for curricular change, better opportunities for students, improvement of teaching, and more effective academic support services. Introduction Department/Program: Aviation/ Professional Pilot Degree program(s): PPILB Person(s) preparing report: Barney King I. Student Outcomes A) List all current SLOs for the Department/Program 1. Students will be highly skilled professional aviators who understand the national airspace system and can interface with all facets of the air traffic control system 2. Students will demonstrate communication skills and apply these skills in the aviation environment 3. Students will understand their responsibility to continue professional and personal development with an emphasis on diversity, ethics and teamwork 4. Students will understand aircraft design, performance, operating characteristics, national and international aviation law and regulations, environmental issues and maintenance. 5. Students will use appropriate aeronautical decision making based on meteorological conditions, human factors and safety. B) Identify applicable K-State Undergraduate Outcome. (knowledge critical thinking communication diversity academic & professional integrity) Program SLOs University-wide SLOs (Undergraduate Programs) Knowledge Critical Communication Diversity Academic / Thinking Professional Integrity 1. X X X 2. X 3. X X 4. X 5. X X Program SLO is conceptually different from university SLOs C) Links to website where the Program SLOs, Assessment summary, and Alignment Matrix for the degree program are located (within two clicks of the Department/Program front page). In fall 11 we have gone to one degree program with six options. The old information is at http://www.salina.k-state.edu/academics/slo/ppilb%20slos.pdf There is no assessment summery for 2009-2010 because we did a self-study for AABI accreditation. Information for the new degree is at http://www.salina.kstate.edu/aviation/slo/professional%20pilot%20mission.pdf II. Assessment Strategies for each SLO that was assessed for this annual report. ( please describe): A) 1) SLO outcome number 1 was assessed using first time pass rates on the FAA knowledge test and first time pass rates for the practical test administered by the FAA or a Designated Pilot Examiner. 2) SLO outcome number 2 was assessed using result of oral presentations in Spring 11 in PPIL 312. B) 1
1) Which and how many students were assessed and when. Between July 1, 2010 and June 20, 2011. There were a total of 98 check rides. On the FAA knowledge tests there were a total of 221 students taking examinations. 2) In PPIL 312 there were a total of 16 students who were graded on a total of 42 presentations. C) Minimum (and advanced if possible) levels for expected student achievement for each SLO. 1) Our goal is that 90% of students will pass on the first try. The FAA grades on a pass/fail basis so no advanced level of expectation is possible. Our goal for knowledge tests is also a 90% 1 st time pass rate. The FAA only provides numbers of passes and average scores so it is difficult to devise an advanced level. 2) Our goal is that 90% of students will get an 8.5 out of 10 on each oral presentation. An advanced goal is that 90% of students would reach a 9.0 out of 10 on their presentations III. Results for Each SLO Assessed A) What percentage of students demonstrate a minimum or higher level of competency? (if possible, what percentage of students achieved at an exceptional level?) Check Ride Attempted Passed Percent 10/11 08/09 07/08 Private 30 21 70.0 91.7 77.8 Instrument 11 7 63.6 67.6 60.7 Commercial 18 16 88.8 75.8 71.0 Multiengine 12 10 83.3 90.6 79.3 Flight Instructor 11 7 63.6 67.6 85.2 Instrument Flight Instructor 16 16 100 89.6 96.3 Total 90 77 78.5 80.8 79.6 1) We were success in check rides at the CFII level and close in Commercial. There was a decrease in Private Pilot success with the remainder of the ratings about the same.. Of the 218 tests students were successful 189 times on the first try for a 85% pass rate. An advanced level could be based on the average scores with a successful level at 80 and advanced at 90%. We noticed a dramatic drop in first time pass rate and average on the FOI. The FAA revamped the test with no notice to the users and adopted many questions that were later discarded. Using 100% pass on the first attempt and 85% as our goal, the Average score of all first time tests are below. P is passed, A is the number attempted. Average Scores A/P Rate 10/11 08/09 07/08 Advanced Ground Instructor AGI 0/0 91 92 Instrument rating IRA 29/25 86 77 80 79 Commercial rating CAX 28/28 100 85 87 86 Instrument Flight Instructor FII 39/36 92 82 84 83 Flight Instructor Airplane FIA 31/24 77 78 89 86 Fundamentals of Instruction FOI 36/27 75 78 94 94 Private pilot rating PAR 54/48 89 84 80 82 Instrument Ground Instructor IGI 1/1 100 86 84 90 221/189 87 2) Of the 42 presentations given, 36 out or 42 or 86 % were rated 8.5 or higher, which exceed our goal of 85 %. 2
B) What does this tell you about student? 1) From the failure we learned that the two greatest areas of failure of private pilots are takeoff, landing and go around and basic instrument maneuvers. For the instrument pilot the greatest area of weakness is instrument approach procedures. 2) Our pass rate on the Fundamentals of Instruction dropped from over 90% to 75% due to the FAA changing the test questions without notifying the public that it was doing so. Other aviation colleges experience the same drop in results that we experience. In looking at the results by topic areas our weakness are: Private Pilot airspace, airport operations and aerodynamics. Instrument pilot navigation, aerodynamics and instrument procedures. Commercial navigation, airport operations and aerodynamics. Flight Instructor Airplane instructional guidelines, navigation, aircraft performance. and Flight Instructor Instrument human factors, instrument procedures and weather. IV Faculty Review of the Assessment Results Describe the process by which program reviewed the results and decided on the actions and/or revisions that were indicated by those results. Faculty were provided the raw data on knowledge test results, FAA practical examinations and oral test results. They were also provided results of our Aviation Accreditation Board International visit. The met to discuss the results and to seek ways to improve our results. Our Academic Achievement Center has tutors but has never had tutors for aviation subjects. We have asked them and they have hired two tutors to help our students who are having difficulties. V Revisions of Assessment Plan A) What changes, if any, were implemented, based on the findings of the assessments? As a result of our AABI accreditation visit, our Aviation Law was weak in international law. A module has been added to resolve this issue. AABI found that our capstone experience was not aligned with program outcomes. The programs outcomes have been changed see VI below and we have added a new course AVT 497 Senior Project. AABI was not satisfied with our after hours tracking procedures. We have added staffing so that we have dispatching coverage on nights and weekends. We developed Educational Objectives that are published on our website in I C) above. In our FAA results if was obvious that navigation, airport operations and aerodynamics are common weak areas in our FAA knowledge testing. These will be stressed in ground school classes. Flight instructors will be briefed on the common failure areas on FAA check rides so that they can stress these areas during flight lessons. B) Revisions of your Assessment Plan or SLOs (if applicable) We have a new curriculum and new assessment measures. See VI below. VI Future Plans Briefly describe the long-range plan to assess all of the outcomes if assessing over a sequence of years. This fall we have gone to a new curriculum and have a new set of outcomes. In the table below are our new outcomes and our plan to assess the 10 outcomes over the next few years. Learning Outcomes Timeline for Assessment of Learning Outcomes Baseline created 2011 2012 2013 Plan and organize flights to demonstrate proficiency at the flight instructor, instrument flight instructor and multiengine rating level. after 2011/12 3
Evaluate meteorological and personal physiological conditions in order to make sound gono go decisions.. Solve problems and employ critical thinking using mathematics and sciences applied to the aviation workplace. Demonstrate an ability to work on multidisciplinary teams. Data Collected Illustrate the principals of crew resource management in the conduct of all flights. after 2013/14 Evaluate the aerodynamic capabilities of aircraft prior to flight based on appropriate performance charts. Data collection after 2013/14 Demonstrate the 4
ability to communicate orally and in writing and produce appropriate visual aids to support presentations. Analyze and interpret information from advance aircraft systems. Distinguishes conditions which could cause an unsafe condition in the air or on the ground and takes appropriate action. Practice appropriate respect for diversity and knowledge of globalization, sustainability, and multiculturalism. Begins after 2013/14 5