Academic Program Review SELF-STUDY APPENDICES

Academic Program Review SELF-STUDY APPENDICES 1

Appendix 1 Academic Program Review Signature Page Title of Program Signatures: Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Program Faculty Member (Print) Signature Title Date Department Chair (Print) Signature Title Date College Curriculum Committee (Print) Signature Title Date General Education Subcommittee (Print) Signature Title Date College Dean (Print) Signature Title Date AVP (Academic Affairs) and ALO (Print) Signature Title Date Graduate Council (if applicable) (Print) Signature Title Date University Educational Policies Committee Signature Title Date (if applicable) (Print) Provost and VP for Academic Affairs Signature Title Date : Academic Program Review Self-Study Appendices 1

Appendix 2 Substitution of Specialized Accreditation Self Study for the Academic Program Review Self Study For programs that undergo professional or specialized accreditation, academic program review is coordinated with the accreditation or re-accreditation cycle. The self study developed for professional or specialized accreditation reviews, provides the essential requirements of academic program review and may be used for this purpose with approval by the college dean. The completed table below and the APR signature page (APR Procedures, Appendix 1) are to be forwarded by the college dean to the AVP/ALO for substitution of accreditation self study for the APR self study. Program Accreditation Agency Date of Self-Study CSU Stanislaus Academic Program Review Component Specialized Accreditation Standard Program Overview 1. Introduction and Historical Context 2. Relationship to Mission (University and College) and Strategic Plan 3. Program Description (with course requirements) 4. Review of Changes since the Last APR/ Accreditation Review Data Review 1. Student Profile and Enrollment Trends (demographics) 2. Faculty Profile 3. Delivery of Instructional Program Location (Turlock, Stockton, Merced, etc.) Distance Education (online, ITV) Scheduling of classes (3 year plan) Justification of additional units 4. Peer Institutions/Benchmarking Commitment 1. Faculty to Student Learning Advising and Mentoring Teaching Research, Scholarship, and Creative Activity 2. Assessment of Student Learning Program Learning Outcomes/Student Learning Outcomes Curriculum Map Student Assessment Student Achievement Resources 1. Facilities (classroom space, equipment) 2. Fiscal (budget review) 3. Technology Resources and Support 4. Library Resources and Information Literacy Implementation Plan 1. External Consultants/Accreditation Review 2. Implementation Action Plan Date of Review Visit Date of Review Report Date of Response to Review Report Note: Appendix 1 (Academic Program Review Signature Page) to accompany Appendix 2. : Academic Program Review Self-Study Appendices 2

GRADUATE EDUCATION ASSESSMENT: FAQS Appendix 3 Graduate Assessment DRAFT Why do we need to assess student learning in graduate programs? 1. Going through the process of developing an assessment plan ensures that your program curriculum is logically designed and reflects the desired student learning outcomes. 2. Assessment data provides evidence that students are meeting faculty expectations for graduates of the program. 3. Assessment is required by WASC, disciplinary accreditation agencies, and for grants such as CEGE, in order to determine if students (or specific sub-groups of students) are meeting student learning outcomes. What are the major steps for developing a graduate assessment plan (linked to APR Procedures)? 1. Program faculty decide on what they want students to learn (Section V-A). 2. Faculty identify the courses that emphasize the various learning outcomes (Section V-B). 3. Faculty identify one or more signature assignments that provide opportunities for students in the program to show their mastery of learning outcomes (Section V-D). Examples of signature assignments could include term papers, exams, projects, and theses. (Note: Steps 1-3 are incorporated into the curriculum map- Section V-B) 4. Faculty develop rubrics for each signature assignment, to facilitate evaluation and comparison across courses or cohorts of students. Ideally, each of the program learning outcomes are included in one or more rubrics. (Section V-E) 5. Assignments are graded by course instructors (and/or additional reviewers). Students scores on signature assignments become the raw data for the direct assessment of student learning outcomes (Section V-D). 6. Data from the direct assessment of student learning (with additional data from indirect assessments, e.g. surveys) are used to evaluate the degree to which students are meeting program learning outcomes, and trends over time. Data are reported in annual reports and used for academic program reviews (Section IX). : Academic Program Review Self-Study Appendices 3

Appendix 3 Graduate Assessment DRAFT GRADUATE EDUCATION: ASSESSMENT PLAN Over a decade ago, the Graduate Council was a leader in creating an assessment approach centered on student learning goals for graduate education and continues its commitment, as a collective governance body, to promoting and evaluating graduate program quality. The Graduate Council created university wide graduate student learning goals that link to the major discipline specific program learning outcomes unique to each graduate program (drafted 1999; approved in 2000). Since that time, the Graduate School has employed various assessment methods for collecting information that has assisted the Graduate Council in its consideration of the quality of graduate programs (see Table 1). These methods collectively contribute to answering the important question of the degree to which our graduate programs achieve their shared goal of educating graduates. The assessment methods described in the Academic Program Review procedures are aligned with and complement the assessment methods used by individual graduate programs. Most importantly, the assessment strategies adhere to the University s Principles for the Assessment of Student Learning (2004) which defines the role of assessment within the institution. The Graduate Council recognizes the complexity of assessment and the significance of designing methods that are multidimensional, meaningful, and oriented toward program improvement and enhanced student learning. Further, the Graduate Council subscribes to the philosophical conviction that the quality of teaching is inextricably connected to the quality of student learning. Thus, while recognizing the importance of student learning outcomes as an important component of program assessment, the Graduate Council avoids reliance on this measure alone as it engages in a critical, comprehensive analysis of the quality of our graduate programs and our graduate students academic achievement. Graduate Student Learning Goals The six overall graduate student learning goals follow. Students will demonstrate 1. advanced knowledge, skills, and values appropriate to the discipline. 2. ability to be creative, analytical, and critical thinkers. 3. ability to work as individual researchers/scholars as well as in collaboration with others in contributing to the scholarship of their disciplines, as appropriate. 4. relevant knowledge of the global perspectives appropriate to the discipline. 5. knowledge of new and various methods and technologies as appropriate to the discipline. 6. advanced oral and written communication skills, complemented, as appropriate to the discipline, by the ability to access and analyze information from a myriad of primary, print, and technological sources. : Academic Program Review Self-Study Appendices 4

Academic Program Review Accreditation *Admission Examinations *Culminating Experience: Thesis, Project, Comp Exam External Consultants Grade Point Average *Student Scholarship Student Awards and Honors Graduate School Exit Survey Graduate Alumni Survey IDEA Course Evaluations Graduate National Survey of Student Engagement Graduate Faculty Survey of Student Engagement Program Approval Processes Appendix 3 Graduate Assessment DRAFT Table 1 displays the alignment between the graduate student learning goals and methods of assessment (both direct and indirect methods). Table 1 Alignment of Graduate Student Learning Goals and University-Wide Assessment Methods Assessment Methods Graduate Student Learning Goals 1. Advanced knowledge, skills, values X X X X X X X X X X 2. Creative, analytical, critical thinking X X X X X X X X X X X X 3. Individual and collaborative scholarship X X X X X X X X X X X 4. Global perspectives X X X X X X X X X X X 5. Methods and technologies X X X X X X X X X 6. Communication skills; source analysis X X X X X X X X X X X X X X *Direct methods. Table excerpted from the CSU Stanislaus Graduate Assessment Plan (2009). The three levels of student learning assessment at CSU Stanislaus are institution-level, program-level, and courselevel. Institution-level graduate learning goals provide a structure for institution-wide learning outcomes assessment. These aims, stated in the graduate student learning goals, collectively specify that all graduate students, regardless of what program they pursue, will be taught and assessed in the knowledge, skills and dispositions defined in these aims. Table 2 illustrates the way in which graduate learning goals could be used as the anchor in an alignment process, whereby program learning outcomes are listed and aligned with graduate learning goals. Similarly, courses that fulfill these learning outcomes are also listed and therefore in alignment with the graduate learning goals. An assessment conducted in a course contributes to determining the level of student competency in the program and towards achieving the graduate learning goals. : Academic Program Review Self-Study Appendices 5

Appendix 3 Graduate Assessment DRAFT Table 2 Alignment of Graduate Student Learning Goals, Program Learning Outcomes, and Courses Graduate Learning Goals Advanced knowledge, skills, and values Creative, analytical, and critical thinkers. Researcher/ Scholar Global Perspectives Knowledge of new and various methods and technologies Advanced oral and written communication /information literacy Program Learning Outcomes e.g., PLO 1 PLO 3 PLO 2,4 PLO 5 PLO 1,5 PLO 6 Courses e.g., Course X Course Y Course Z Course A Course B Course C Graduate Council: Reviewing and Reporting on Graduate Assessment Results Graduate Council reviews all graduate-level APRs. Information is collected from the programs on student achievement on the six learning goals. Adapted from Draft Institutional Plan for the Assessment of Graduate Programs (2012). : Academic Program Review Self-Study Appendices 6

Appendix 4 Academic Program Review Timeline TARGET DATE By February 1 By March 1 By March 15 March 16 May 29 March 16 May 29 March 16 (Year 6) February 1 (Year 7) YEAR 6 ACTIVITY AVP (Academic Affairs) notifies college deans and department chairs/program administrators the programs to be reviewed two years prior to the completion date of the self study, recommendations, and implementation plan. Department chair/program administrator request from the college dean that the program be subject to an external evaluation. An external consultant to be invited to assist in the self-study phase of the academic program review process. AVP (Academic Affairs) conducts a program review workshop(s) with department chairs/program administrators and program faculty to discuss the academic program review process and discuss data provided by institutional research, as required for the academic program review. Department chair and dean identify process and timeline for milestones for completion and identify/arrange for external reviewers (as appropriate). Program faculty and department chair begin draft review of data and begin draft of self study. Program faculty and department chair conduct the self study and complete the self-study document, including recommendations and a preliminary implementation plan. TARGET DATE By February 1 February 1 27 February 15 April 30 February 15 April 30 By April 30 By April 30 April 30 May 29 May 29 June 30 By June 30 YEAR 7 ACTIVITY Department chair/program administrator submits the self study and supporting materials to the college dean. College dean submits self study to external reviewers (as appropriate). College governance committee(s) reviews the self study, requests additional materials as needed, summarizes findings, and forwards the self study to the department chair/program administrator. General Education Subcommittee reviews the General Education portion of the self study, summarizes findings, and forwards the recommendations for recertification of the GE curriculum (lower- and upper-division) to the department chair/program administrator. College dean forwards the self study to the Office of Academic Programs. Office of Academic Programs forwards the self study to the UEPC (if requested) and/or to the Graduate Council (for master s and post-baccalaureate programs). UEPC and/or Graduate Council (as appropriate) reviews the self study, summarizes the findings, and forwards the document and findings to the department chair/program administrator and college dean. College dean finalizes self study to include recommendations from external reviewer(s) (if applicable); responses from the department (if any); recommendations from the college governance committee(s), UEPC, and/or Graduate Council; and dean s recommendation for program continuance, continuance with conditions, or program discontinuance. College dean submits to the AVP the self study; recommendations from external reviewer(s) (if applicable); responses from the department (if any); recommendations from the college governance committee(s), UEPC, and/or Graduate Council; and dean s recommendation for program continuance, continuance with conditions, or program discontinuance. : Academic Program Review Self-Study Appendices 7

Appendix 4 Academic Program Review Timeline TARGET DATE September October October November By December 1 By December 15 By January 15 ONGOING YEAR 7 ACTIVITY continued Office of Academic Programs schedules a meeting to include the program representative(s), the department chair/program administrator, the college dean, the AVP, and the provost to discuss the results of the academic program review and the preliminary implementation plan. Department chair/program administrator submits to the college dean a final implementation plan that identifies resource needs consistent with the recommendations of reviewing committees and consistent with the college mission and strategic plan. Within three weeks, the college dean submits the final implementation plan to the AVP. Provost issues a letter indicating final determination of program continuance and additionally may require progress reports and a timeline related to specific elements of the final implementation plan. Office of Academic Programs archives the academic program review documents and posts on the web (program faculty s final implementation plan and provost s recommendation for program continuance/discontinuance). AVP provides a summary of academic program reviews to the Board of Trustees. College dean incorporates the results of the academic program review into the college s strategic and budget planning processes and forwards to the provost as part of the regular planning and budgetary processes within academic affairs and within the university s strategic planning processes. Department chair/program administrator submits annual reports (due September) based on program review implementation plan to college dean. ACCREDITED PROGRAMS Accredited programs will follow the applicable timeline steps as they apply to the accrediting agency s due date and timeline for the program s self study. After completion of the accreditation review, and upon receipt of the accrediting agency s determination letter, the Department chair/program administrator requests of the college dean a substitution for the academic program review document. College dean determines whether the accreditation review process fulfills all or a portion of the academic program review in accordance with any CSU or CSU Stanislaus mandated requirements and communicates decision to the department chair/program administrator. College dean submits completed substitution for the academic program review documents to the Office of Accreditation. Office of Accreditation schedules a meeting to include the program representative(s), the department chair/program administrator, the college dean, the AVP, and the provost to discuss the results of the accreditation self study and the recommendations. If periodic reporting is not required by the accrediting agency, an implementation plan will be developed for the university s annual reporting process. : Academic Program Review Self-Study Appendices 8

E.g., Course A Course B Course C Course D Course E Course F Continue Appendix 5 Curriculum Map Template Core Courses Program Learning Outcomes E.g. Written Communication Skills Oral Communication Skills I I E Information Literacy Critical Thinking Quantitative Literacy I E R E E KEY: Outcomes assessment data not collected for program-level analysis Outcome assessment data collected for program-level analysis NOTE: In the curriculum map, the level at which the outcome is addressed is identified. In this case, the following scale is used: Introduced (I), Emphasized (E), and Reinforced (R). Other options include: Introduced (I), Developed (D), and Mastered (M) Introduced (I), Practiced (P), Demonstrated (D) Introduced (I), Reinforced (R), and Mastered (M) Basic (B), Intermediate (I), Advanced (A) : Academic Program Review Self-Study Appendices 9

External Consultant s Letter of Invitation DATE Dr. XXX XXX Address Dear Dr. XXX: Appendix 6 External Consultant Review I am writing to invite you to serve as external consultant for XXX program (BA, BS). As part of CSU Stanislaus assessment effort, every academic program undergoes an Academic Program Review every seven years. The review is intended to help us identify a program s strengths and areas in which it could improve; determine the program s education effectiveness by assessing student learning; and provide data for informed planning. Dr. XXX, Professor of XXX and Department Chair has suggested you as someone who could help us with that task. The role of the external consultant is to provide an objective assessment of the quality and effectiveness of the academic program, resources, and operations based on the program s self-study and questions asked by program faculty and administration. We would ask you to read the XXX program self-study; visit CSU Stanislaus sometime this fall to meet with members of the department and administration as well as students; and write a report based on the self study and interviews, responding to specific questions that faculty and administration might ask as well as noting what a program is doing well and making overall recommendations for quality enhancement. I have attached three documents that should help address questions that you might have about the review: 1) the template for the external consultant s report; 2) a typical schedule; and 3) a document describing the purpose of the review. The XXX Department has prepared some specific questions that I will send should you accept this invitation. In addition, we would send the departmental study and additional questions that I would ask you to address. CSU Stanislaus will pay your travel expenses and an honorarium of $250 per day. I will follow up this invitation next week with a phone call to see if you are able to do this and to answer any questions that you might have; please feel free to call or write in the meantime. If you are able to do this, I will ask Mr. XXX, my administrative assistant, to call you to schedule your visit and help you make travel arrangements. We would be very pleased if you are able to do this. Thank you for considering the invitation. I look forward to hearing from you. Sincerely, XXX Dean, College of XX : Academic Program Review Self-Study Appendices 10

Appendix 6 External Consultant Review External Consultant s Sample Schedule Program Date Time Interview / Meeting Location Date Evening Dinner with Faculty / Dean / Consultant Restaurant 8:30 9:30 am Program Department Chair Tour of Campus / Facilities 9:30 10:00 am Meet with Dean 10:00 10:30 am Meet with AVPAA 10:30 11:00 am Meet with Faculty 11:00 am Noon Classroom Observation Date Noon 1:00 pm Lunch with Students 1:00 3:00 pm Meet with Faculty 3:00 4:00 pm 4:00 4:30 pm Meet with Alumni Meet with Service Learning / Librarian / Institutional Research 4:30 5:00 pm Meet with Faculty to deliver Preliminary Impressions and Report 5:00 pm Departure **include time for external consultant to meet with Human Resources and sign paperwork (if employed by the CSU) : Academic Program Review Self-Study Appendices 11

External Consultant s Report Template Appendix 6 External Consultant Review Program External Consultant s University Date This template is provided to assist External Consultants in the completion of their report. External Consultants are asked to write a report that: Includes an executive summary of general comments, singles out features of the program that merit commendations, and makes recommendations for improvement. Varies in length between three and five pages. Is completed within two weeks of the visit. 1. Executive Summary Provide a brief executive summary of major findings for this program. Include: General observations and comments on the program and curriculum, quality of student learning and the achievement of student learning outcomes, the implementation plan, faculty, students, facilities, and resources Reponses to questions posed by faculty 2. Commendations Provide comments about what the program is doing well. Note suggested topic areas below. 3. Recommendations Provide comments to guide future direction for faculty to use to improve student learning. Provide evaluative feedback that would improve any aspect of the program and recommendations that require no new resources as well as those that do. The report may note recommendations that have been shown to be effective elsewhere. Note suggested topic areas below. Educational Effectiveness Topic Areas for Commendations and Recommendations sections: Provide feedback / suggestions on any learning outcome. Analyze / evaluate direct and indirect evidence of student learning Offer suggestions to improve the assessment process Evaluate assessment plan Evaluate assessment impact : Academic Program Review Self-Study Appendices 12

Appendix 6 External Consultant Review EXTERNAL CONSULTANT FUNDING REQUEST FORM Programs may apply for up to $1,000 to assist with an external consultant s fees and travel expenses. As noted in the Academic Program Review Procedures, the college dean and chair of the department coordinate the external consultant s travel arrangements and visit schedule, in accordance with University travel policy. A consultant contract is issued to the external consultant (normally $250 per day), plus transportation and one-night lodging, as required. The payment and refunds are processed upon receipt of the written report from the external consultant and documented accommodation and travel costs, as previously approved. Departments will be reimbursed for approved external consultant expenses upon receipt of a copy of the External Consultant s report to the Office of Assessment. For additional information on the required qualifications and selection of an external consultant, see Academic Program Review (APR) Procedures Section VI: External Consultant Review. Completed request forms can be submitted to the Office of Assessment - MSR 376. Today s Date College Department/Program Primary Contact Phone Email Budget Summary Item Description e.g., travel Total Budget Requested : Academic Program Review Self-Study Appendices 13

Appendix 6 External Consultant Review Timeline Action Month / Date Person Responsible Signatures Department Chair Date Dean Date AVP (Academic Affairs) Date : Academic Program Review Self-Study Appendices 14

Appendix 7 Seven Year Implementation Plan Template College: Program: Next APR Year: Mission Statement: [Enter your program Mission Statement here] Program Learning Outcomes: PLO 1: PLO 2: Etc. Program Maintenance Outcomes: PMO 1: PMO 2: Etc. Where are these outcomes published? [e.g., University Catalog, Department website] Please attach the most current program curriculum map. SEVEN YEAR IMPLEMENTATION PLAN TEMPLATE INITIATIVE/TASK Program Learning Outcome (PLO) PLO 1: APR CYCLE YEAR ASSESSMENT METHODS/ACTIVITIES ASSESSMENT PROCESS: COLLECTION/ANALYSIS/ DISCUSSION RESOURCES NEEDED (IF APPLICABLE) PLO 2: PLO 3: PLO 4: Program Maintenance Outcome (PLO) PMO 1: PMO 2: PMO 3: PMO 4: : Academic Program Review Self-Study Appendices 15

Appendix 8 California State University Stanislaus Provost's Academic Program Review Meetings This template provides an overview of the provost's Academic Program Review meeting. Participants include the department chair, program coordinator and/or faculty (as determined by department chair or dean), dean, and the associate vice president (AVP). INTRODUCTIONS (DEAN) 1. Dean's introductions of meeting participants. OVERVIEW (DEAN AND DEPARTMENT CHAIR) 2. Dean's overall evaluation of program and key issues identified by the college's committee and dean. 3. Department chair's overview of major findings/issues. PROGRAM QUALITY (PROVOST) 4. Provost's questions/discussions regarding the self-study's findings. 5. Overall conclusions about program quality and assessment of student learning. 6. Implementation Plan: a. Program Learning Outcomes b. Program Maintenance Outcomes NEXT STEPS (AVP) 7. Explanation of next steps a. Department chair completes final Implementation Plan and submits to the dean; dean submits electronic document to AVP (within 2 weeks following meeting). b. Upon receipt and review of the final Implementation Plan, Provost issues letter indicating final determination for program continuance. EVALUATION OF THE ACADEMIC PROGRAM REVIEW PROCESS (AVP) 8. In what ways did the faculty find this academic program review process helpful for program improvement? 9. What are your recommendations for improving the academic program review process? a. Review Criteria, especially student learning b. Internal/External review components c. Institutional research/assessment data d. Timeline e. Department/college review process and report f. Other CLOSING COMMENTS (PROVOST) : Academic Program Review Self-Study Appendices 16

Appendix 9 Annual Report Template DATE DUE: September 20, 2012 College: Program: Next APR year: Program Learning Outcomes: PLO 1: PLO 2: Etc. Program Maintenance Outcomes: PMO 1: PMO 2: Etc. Assessed Outcomes Assessment Methods Data Reviewed and Findings Actions (List PLO# and/or PMO# for outcomes assessed during the previous AY) (Describe the assessment methods used this year and indicate the targeted PLO by #) (Provide a description of the data reviewed and a summary of the findings. Describe the process for evaluating/analyzing the findings) (Describe implemented or planned actions based on findings) Approvals: Department Chair/ Program Director Date College Dean Date : Academic Program Review Self-Study Appendices 17

Appendix 10 Degree Audit Information Department Program Title and Degree Line Proposed Program (# of units) Description 1 51 University general education requirements 2 Prerequisites to the major 3 Upper-division (major requirements) 4 WP course (if not required in the major) 5 Other (if applicable) 6 TOTAL minimum units required (add lines 1 through 5) 7 University elective units (subtract line 6 from line 8) 8 120 TOTAL UNIT DEGREE REQUIREMENTS 9 10 11 12 WP course required in the major Course prefix and number: Lower-division prerequisite course(s) that may be applied toward GE Course prefix, number, units, area: Course prefix, number, units, area: Course prefix, number, units, area: Course prefix, number, units, area: Course prefix, number, units, area: Course prefix, number, units, area: TOTAL double-counted courses (add lines 9 and 10) TOTAL units taken (subtract line 11 from line 8) Prepared by Approved by Date Date : Academic Program Review Self-Study Appendices 18