Direct & Indirect Measures of Student Learning

Transcription

1 Direct & Indirect Measures of Student Learning Direct Measures of Student Learning Type Definition Considerations: Appropriate to: Capstone Course/Final Project Comprehensive Post Test (written and/or performance) Portfolio Internship Clinical Evaluation Credentialing Exam Results Standardized Tests Comprehensive review and assessment of highest level of content from all courses in program. Synthesis of learning in entire program of study Tests skills/knowledge in program of study as well as general education learning. Tests at level of learning for entry-level employees. Test questions typically are NOT true/false, multiple choice or matching, but rather essay, troubleshooting, evaluative, higher-level thinking and synthesis. Psychomotor outcomes evaluated through demonstration of skills/performance. Compilation of projects that demonstrate learning outcomes from program area as well as general education. Includes student rationale regarding project inclusion in portfolio. Student demonstrates skills/knowledge in all program and general education learner outcomes Occurs in specific occupational setting Results obtained regarding student performance on state and/or national credentialing exams provides info about program performance. Comparison with other institutions Assessment requires additional faculty time. Project may require time and cost of project evaluators. Time and cost involved in local development of tests. Psychomotor skills demonstration requires additional faculty time to administer. Evaluation by external evaluators is preferred. Internship supervisor time required to monitor and work with employer regarding ou5comes and evaluation criteria and standards. Any program especially those where students will take a credentialing exam Programs with certification exams and/or accreditation standards that specify skills testing and knowledge testing. Cognitive programs best for paper/pencil, whereas skills program best for performance. Project-based coursework is conducive to portfolio creation. Variety of mediums can be included. Any employment preparatory program.

2 Indirect Measures of Student Learning Type Definition Considerations: Academic/Exit Survey (approx. one month prior to graduation) Alumni/Graduate Survey (approx. 4-6 months postemployment) Employer Survey (approx. 4-6 months postgraduation) Exit interviews of graduates and focus groups Participation/persistence Data Students rank their ability to perform the program outcomes. Graduates rank their ability to perform as an entry level employee in their discipline. Employers rank the graduates ability to perform as an entry level employee in their discipline. Provides info about student experience about courses, instructional methods Enrollment/graduation rates Responsibility for development and data collection Responsibility for development & data collection Responsibility for development & data collection 11/17/06; rev. 1/10/07

3 Northland Community and Technical College Program Assessment Plan Liberal Arts Liberal Arts Program Assessment Plan Academic Year: Discipline: Date Submitted: Part I: Assessment Plan Program Learner Outcomes (Insert the appropriate Liberal Arts Program Learner Outcome/s for your discipline) Assessment Measure (List the direct assessment method for the Learner Outcome) Performance Target/s Submitted by: Part II: Effectiveness Report Year of Data (What year was data collected?) Findings (What did you learn from your assessment? Include a summary of Assessment Data collected.) Action Taken or To Be Taken (How are the results being used to improve the instructional program?) 5/1/2009

4 Northland Community & Technical College Program Assessment Plan Career Programs Career Program Assessment Plan Academic Year: Program: Date Submitted: Number of Credits: Degree: Submitted by: Campus/es: Part I: Assessment Plan Part II: Effectiveness Report Assessment Question #1: Have students learned? Goal: To provide quality competency-based technical education and general education to meet the students needs of ensuing achievement of knowledge, skills, and professional behaviors and attitudes required of entry-level employees. **Assessment Measure 1 direct measure as well as the student exit survey and the employer survey are required in all program assessment plans. Direct Measure: Performance Target Target: Year of Data (What year was data collected?) Findings (Is the Target met? What are your findings relative to the Target stated?) Action Taken or To Be Taken (How are the results being used to improve the instructional program? If the target was not met, why? What can be done to improve results?) Indirect Measure: Exit Survey Target: Indirect Measure: Employer Survey Target: Indirect Measure: Target: 5/1/2009

5 Northland Community & Technical College Program Assessment Plan Career Programs Assessment Question #2: Have students graduated? Goal: To contribute to workplace capacity and college competitive advantage by enrolling and graduating a sufficient number of students within a reasonable time frame to make the program economically and academically feasible. Assessment Measure Performance Target Year of Data Assessment Measure Program Graduation Rates Target: (What is your goal for how many students will graduate?) Findings Assessment Question #3: Are students employed/ continuing education? Action Taken or To Be Taken Goal: To respond to the human resource needs of employers by providing competent entry-level employees. Assessment Measure Placement Rates Target: (what is the goal for the number of students employed or continuing their education?) Number of students employed: Number of students continuing education: Other findings: 5/1/2009

6 Northland Community & Technical College Program Assessment Plan Career Programs ** Process: All program faculty are to be involved in the development of the Program Assessment Plan. I. For each Assessment question asked, list the evaluation methods used to determine progress towards your program outcomes. Include information regarding the timelines for data collection. II. Implement the Program Assessment Plan, conducting the indicated assessments and collect the data required according to the Plan. III. Include data/findings for each of the identified measures and a list of what actions will be taken based on the findings (i.e., changes to the curriculum, instructional delivery, etc.). Submit completed Assessment Plan to your Academic Dean by June 1 st unless another date has been agreed to for your program. The Program Assessment Plan must include at least one direct indicator of academic achievement in Student Learner Outcomes identified for the Program. Exit survey results are a required indirect measures. Direct measures require students to demonstrate what they know or can do with their knowledge. These include many of the assessment types currently in place in the College s courses (i.e., Pre- and/or Post Tests, Portfolios, Capstone Courses, Capstone Projects, Course-based Assessments, Certification/Credentialing Tests, etc.) Indirect measures include self-reporting assessments such as surveys (e.g., surveys that students, alumni, or employers complete, as well as evaluations done by external reviewers such as accreditation agencies). 5/1/2009

7 Assessment Component Student placement in areas of reading, math, and writing Direct Assessment of student academic achievement on Institutional Learner (Program) Outcomes (formerly referred to as Core Competencies ) Student selfassessment of academic achievement on Student Learner (Program) Outcomes (indirect assessment) Assessment Activity/ Measurement Tool Basic Skills Test (AccuPlacer) or equivalent that demonstrates collegereadiness Program-Level Assessment Matrix identifies the various means of assessing student achievement: For example-- Pre/Post Tests Portfolios Capstone courses, including internships Capstone projects Course-based assessments CATS Responsibility Admissions Personnel Faculty will ensure that such assessment is part of the overall Student Achievement Assessment Matrix Assessment Focus All applicants Students in all programs Academic Survey Program Faculty Every student planning to graduate and/or transfer at the end of a student s final semester Frequency Annual results compiled into reports to appropriate College departments Annual results compiled into reports Upon graduation semester (student s final semester) Distribution of Results Reports to Faculty, Deans, Provost and Assessment & Reports to faculty, Deans, Provost and Assessment & and applicable Advisory Boards Reports to Faculty, Deans, Provost and Assessment &

8 Student academic achievement on workplace skills and knowledge as outlined by state and/or national occupation tests Course/instructional Assessment Assessment of internship experience (as applicable) by Business/Industry Panel (as applicable) Certification/credentialing testing National or state credentialing exams Course Survey Internship Evaluation Job Task Analysis Program Faculty Academic Deans & Faculty Individual Program Faculty (as applicable) Program Faculty and Academic Deans All eligible Programs At least one course per faculty every semester; every course for probationary faculty All applicable programs Programs Annual results compiled into reports Each semester Annual results compiled into report Annual results complied into report Reports to Faculty, Deans, Provost, and Assessment & Committee, and applicable Advisory Boards Reports to Individual faculty and Academic Dean Reports to Faculty, Deans, Provost and Assessment & and applicable Advisory Boards Program Faculty, Deans, Provost and Assessment & and applicable Advisory Boards

9 Academic Program Review (comprehensive five-year review of entire Program) Study of employers perceptions of graduates skills Study of Alumni perceptions of their NCTC education MnSCU Graduate Follow-Up (employment verification) Enrollment data, analysis, retention data, assessment data, financial information, etc. Employer Survey of Institutional and Program Learner Outcomes Alumni Survey of Institutional and Program Learner Outcomes Graduate follow-up survey for job placement, employment rate, etc. Faculty, Academic Deans, Provost, Student Services, and Chief Financial Officer Program Faculty & Director of Employment Development & Placement Services Program Faculty Director of Employment Development and Placement Services Programs Sample of employers of NCTC graduates Sample of NCTC alumni who graduated All recent graduates Every five years on a rotating schedule Annual or biannual results compiled into reports Annual or biannual results compiled into report Each semester; results compiled into annual summary report Program Faculty, Deans, Provost and Assessment & and applicable Advisory Boards Program Faculty, Deans, Provost and Assessment & Program Faculty, Deans, Provost and Assessment & Faculty, Deans, Provost and Assessment &

10 PROGRAM: Assessment Component Student placement in areas of reading, math, and writing Direct Assessment of student academic achievement on Institutional Learner (Program) Outcomes (formerly referred to as Core Competencies ) Student selfassessment of academic achievement on Student Learner (Program) Outcomes (indirect assessment) Student academic achievement on workplace skills and knowledge as outlined by state Assessment Activity/ Measurement Tool Responsibility Assessment Focus Frequency Distribution of Results

11 and/or national occupation tests Course/instructional Assessment Assessment of internship experience (as applicable) by Business/Industry Panel (as applicable) Academic Program Review (comprehensive five-year review of entire Program)

12 Study of employers perceptions of graduates skills Study of Alumni perceptions of their NCTC education MnSCU Graduate Follow-Up (employment verification)

13 ASSESSMENT OF STUDENT INSTITUTIONAL LEARNER OUTCOMES MATRIX (OPTIONAL) Program Name: Date Submitted: Campus Location(s): Academic Award: Program Contact Person: Number of Required Credits: Process: Listed in the first column are the NCTC Institutional Learner Outcomes. Using the rubrics shown, identify the level at which each Institutional Learner Outcome is assessed in each course included in the program. If an outcome is NOT assessed in a particular course, write 0 rather than leaving the box blank. Then for each post-program assessment administered for the program, also rate the assessed level of each outcome. Each program must include at least one comprehensive, direct post-program assessment. (NOTE: If the post-program assessment is a capstone course already rated previously, include the same ratings in the Post-Program column.) Rubrics: 1 = minimal, informal assessment at Learning Level 1 (i.e., define, imitate, identify); 2 = assessed through a minimum of at least one formally evaluated activity or 3-4 minor activities at Learning Level 2 (i.e., explain, demonstrate with guidance, perform); 3 = assessed formally through more than one major activity as well as through other informal and formal assessments at Learning Level 3 (i.e., analyze, evaluate, synthesize). Institutional Learner Outcomes: Listed below are the Institutional Learner Outcomes established for NCTC. The level of assessment of these outcomes must be indicated in appropriate general education courses, in addition to (not in lieu of) any technical courses in which these outcomes are assessed. Program Courses: List by course number and credits. Add columns as needed. Course Course Post-Program Direct Assessment Pre/Post Test (Commercial/Local) Comprehensive Project Portfolio Capstone Internship Capstone Course Certification/Licensure Exam Other Post Program Assessment Foundational Skills Thinking Skills Global & Civic Responsibility Applied & Information Technology Personal Development Credit is given to Madison Area Technical College for this format. *NOTE: This is an evolving piece of collaboration by NCTC faculty to acknowledge and articulate institutional learner outcomes that all students should obtain, to some degree, during their college career. This list represents skills and aspects of personal growth that students who are completing certificates, diplomas, or degrees at NCTC have met or have exercised. Various expectations of the degree to which these outcomes will be met may exist. 11/17/06

14 ASSESSMENT OF STUDENT PROGRAM LEARNER OUTCOMES MATRIX (OPTIONAL) Program Name: Date Submitted: Campus Location(s): Academic Award: Program Contact Person: Number of Required Credits: Process: List all Student Learning Program Outcomes in the first column. Then using the rubrics shown, identify the level at which each Student Learning Outcome is assessed in each course included in the program. If an outcome is NOT assessed in a particular course, write 0 rather than leaving the box blank. Then for each post-program assessment administered for the program, also rate the assessed level of each outcome. Each program must include at least one comprehensive, direct post-program assessment. (NOTE: If the post-program assessment is a capstone course already rated previously, include the same ratings in the Post-Program column.) Rubrics: 1 = minimal, informal assessment at Learning Level 1 (i.e., define, imitate, identify); 2 = assessed through a minimum of at least one formally evaluated activity or 3-4 minor activities at Learning Level 2 (i.e, explain, demonstrate with guidance, perform); 3 = assessed formally through more than one major activity as well as through other informal and formal assessments at Learning Level 3 (i.e., analyze, evaluate, synthesize). Student Learning Outcomes: List all Student Learning Outcomes established for this program. Asterisk those outcomes that include foundation general education. Level of assessment of these outcomes must be indicated in appropriate general education courses, in addition to (not in lieu of) any technical courses in which these outcomes are assessed. Add rows as needed. Program Courses: List by course number and credits. Add columns as needed. Course Course Post-Program Direct Assessment Pre/Post Test (Commercial/Local) Comprehensive Project Portfolio Capstone Internship Capstone Course Certification/Licensure Exam Other Post Program Assessment Credit is given to Madison Area Technical College for this format. *NOTE: This is an evolving piece of collaboration by NCTC faculty to acknowledge and articulate institutional learner outcomes that all students should obtain, to some degree, during their college career. This list represents skills and aspects of personal growth that students who are completing certificates, diplomas, or degrees at NCTC have met or have exercised. Various expectations of the degree to which these outcomes will be met may exist. 11/17/06