PROGRAM REVIEW HANDBOOK

PROGRAM REVIEW HANDBOOK 2015 20 Quality Assurance Funding Reporting Cycle Austin Peay State University August 13, 2018

2 P age

Table of Contents What is Program Review? 4 Why is Program Review important? 4 What is the schedule for the Program Review Process? 5 How do we identify potential reviewers? 7 What is included in the self study? 8 What additional information might the reviewers need? 8 How do the reviewers assess the program? 8 What are the responsibilities of the program reviewer(s)? 9 How is the site visit scheduled? 9 Appendices Appendix A: Sample Email Invitation for External Reviewer 10 Appendix B: Sample Itinerary of Program Review Visit 11 Appendix B: Associate Program Review Self Study Report Template 13 Appendix C: Baccalaureate Program Review Self Study Report Template 19 Appendix D: Graduate Program Review Self Study Report Template 25 Appendix E: THEC Program Review: Certificate and Associate Programs Rubric 31 Appendix F: THEC Program Review: Baccalaureate Programs Rubric 34 Appendix G: THEC Program Review: Graduate Programs Rubric 37 3 P age

What is Program Review? Academic program review is a peer review process designed to improve the quality of the university s academic programs. Program reviews provide a systematic method to evaluate quality, productivity, and need, both in the university and across the state and region. Historically, Austin Peay State University participated in Academic Audit, which was a similar process where reviewers were provided by the Tennessee Board of Regents. The 2018 19 academic year marks the first year since 2006 that Austin Peay has elected to use Program Review for its evaluation process. Each academic program, not accredited by a recognized agency which accredits programs for that field and degree level, must participate in the Program Review process. Programs must undergo an evaluation once in every five year reporting cycle as part of the Tennessee Higher Education Commission s (THEC) Quality Assurance Funding process. During the designated year of review, an academic program will collaborate with the Office of Institutional Effectiveness and Assessment and their respective college dean to complete a selfstudy report and site visit. After reading the report and participating in the site visit, reviewers of the program will compile a narrative report that includes recommendations for improvement. Programs create action plans based on these recommendations and present the outcomes of the Program Review to the Provost and senior administration involved in implementing changes. Ongoing tracking of recommendations and outcomes continues until the next scheduled review. The program review cycle provides the vital link that enacts improvements brought to light in self study and peer review processes. The Office of the Institutional Effectiveness and Assessment coordinates all reviews. This office serves as a resource for you as work begins on your self study. Decision Support and Institutional Research can also offer historical data needed for this report. Why is Program Review important? Program review is an important tool designed to help you identify the strengths and weaknesses of your program so improvements can be recommended and implemented. The follow up process after program review makes sure these improvements are applied. Program Review and Accreditation serves as one of five Student Learning and Engagement standards of THEC Quality Assurance Funding. Each year, the university receives recommended points for each of these standards as well as a standard for Student Access and Success. The Academic Programs: Accreditation and Evaluation standard offers Austin Peay the opportunity to earn 25 of the 100 points awarded annually by THEC. 4 P age

What is the schedule for the Program Review Process? APSU Program Review Timetable 2018 19 Timeframe Activity Responsibility Programs Health & Human Performance (BS & MS), Psychological Science (BA/BS), Political Science (BA/BS), Liberal Arts (AS), English (BA/BS & MA) and Industrial Organizational Psychology (MSIO) March 2018 Attend APSU Program Review Orientation VP/AVPAA, Dean, Chair, PR Team Lead* April 2018 Lead an organizational meeting with departments to begin self study Dean process April 2018 Begin self study process PR Team Lead Pre Semester Week August, 2018 Lead a program meeting to discuss program data PR Team Lead and Program Faculty PR Team Lead August 17, 2018 Submit data request to DSIR for needed data not provided on the DSIR web site** September 17, 2018 Send reminder to PR team lead, Chairs & Deans regarding APSU self VP/AVPAA study template October 2018 Attend APSU Academic Audit presentations from 2017 2018 cohort VP/AVPAA Dean, Chair, PR Team Lead November 9, 2018 Submit recommendations by each self study team for reviewers (2) with PR Team Lead CVs & preferred Program Review team site visit date to Quality Assurance Coordinator November 16, 2018 Submit draft of self study to Chair for review and feedback PR Team Lead November 29, 2018 Return draft to PR Team Lead with suggestions for revision Chair December 3 13, 2018 Incorporate Chair feedback and revisions, reaching consensus with Chair PR Team Lead & Chair December 14, 2018 Submit revised draft to Chair for signature on or prior to Dec. 14 PR Team Lead December 17, 2018 January 4, 2019 Submit revised draft with Chairs signature to Dean for review and feedback; for graduate programs being audited, a revised draft also submitted to the Dean of College of Graduate Studies Return draft with suggestions for revision to the PR Team Lead; Dean of College of Graduate Studies returns draft with feedback to graduate program Incorporate Dean s feedback and revisions, reaching consensus with Deans PR Team Lead Dean & Graduate Dean January 7 9, 2019 PR Team Lead & Dean, Graduate Dean January 10, 2019 Submit revised draft to Dean for signature on or prior to Jan. 10 PR Team Lead January 11, 2019 Submit revised draft of self study with Chair and Dean signatures to VP/AVPAA for review and feedback PR Team Lead January 9 11, 2019 Finalize On site Visit Team schedule and hotel reservations PR Team Lead 5 P age

January 15, 2019 Return report with suggestions for revision to the PR Team Lead VP/AVPAA January 15 17, 2019 Incorporate VP/AVPAA feedback and revisions, reaching consensus with VP/AVPAA PR Team Lead & VP/AVPAA January 18, 2019 Submit final report to VP/AVPAA for signature on or prior to Jan. 18 PR Team Lead January 18, 2019 Submit final self study to Provost for review with Chair, Dean, and VP/AVPAA signatures**** VP/AVPAA January 25, 2019 Return final self study with Provost signature to PR Team Lead Provost January 25, 2019 January 25, 2019 January 28, 2019 February 2019 March 11 April 19, 2019 Within 30 calendar days after site visit Within 1 week of receiving all written reports from reviewers Within 2 Weeks after receiving written report from reviewers September 13, 2019 October, 2019*** Submit final self study, on site visit schedule, and hotel reservations to Quality Assurance Coordinator for distribution to External Reviewers Send Outlook meeting invitations to Dean, VP/AVPAA, and others who should attend the introductory session and the exit meeting Submit to External Reviewer(s) the final self study, on site visit schedule, and hotel reservations Host a Pre Site Visit prep meeting with deans, department chairs, and PR Team Lead Site visits for all programs Obtain written report from reviewers Submit signed rubrics, reviewers narrative reports and CVs of reviewers to THEC staff Discuss preliminary observations concerning the program, criteria ratings, and recommendations for improvement during exit meeting Prepare written responses to the reviewer s report and ratings; the response should address the observations and recommendations in the reviewers report and identify appropriate actions to be taken; submit to Dean and Quality Assurance Coordinator Present response to PR report, rating and recommendations (in a meeting with senior administration) PR Team Lead PR Team Lead Quality Assurance Coordinator Chairs, PR Team Leads, Deans, Graduate Dean, QA Coordinator VP/AVPAA Chair and PR Team Lead Quality Assurance Coordinator Quality Assurance Coordinator Chair, PR Team Lead, VP/AVPAA, Dean, Graduate Dean Chair, PR Team Lead and Program Faculty Chair, PR Team Lead Ongoing Program improvement activities Implemented by program *PR Team Lead: Program Review Team Lead, faculty member of department leading the program audit **www.apsu.edu/dsir/data *** Italicized dates are estimated dates ****PR Team Lead be available for discussion of report with Provost 6 P age

How do we identify potential reviewers? Two to three external reviewers (from outside the state of Tennessee) and two or three internal consultants should be identified for this process. Please contact potential reviewers to ask if they are willing to be considered and able to serve. Once you have established those that have agreed to serve, provide a list of these reviewers to the Quality Assurance Coordinator as early in the academic year as possible, but no later than Friday, November 9. All reviewers must meet the qualifications listed below. The VP/AVPAA will select from your list the external reviewers who will conduct the review, based on their credentials and availability. The external reviewers must be professionals in the field of study under review. The reviewers chosen cannot have personal or professional affiliation with members of APSU s faculty within the program under review and avoid any conflict of interest. (See below for further specifications.) The Vice Provost and Associate Vice President for Academic Affairs (VP/AVPAA) and THEC will have final approval of the review team. When you provide the list of potential reviewers, please include contact information (including email address), and a copy of their vita (or links to web versions). External Reviewers must: 1. Be professionals in the field under review; 2. Hold a terminal degree; 3. Hold an academic position, preferably at a regional public university comparable to APSU; in some cases a practicing professional in the field or a retiree is an appropriate substitute; 4. NOT be APSU graduates; 5. NOT have active or previous professional or personal affiliations with faculty or staff in the department to be reviewed, or with other reviewers (co author, classmate, professor/student, former colleague, etc.); It is recommended that each of the following qualifications is held by at least one reviewer: 6. Department Chair or coordinator experience; 7. Training/experience as a program reviewer. Internal Consultants Two to three internal consultants are included on every review team. One should be from within the same college as the program under review; one should be from outside that college. We give preference to consultants from programs that will be reviewed in coming years, in order to mentor those future reviews and provide a fresh perspective. The internal consultants can provide important campus related information to external reviewers, but they are still key members of the reviewing team, providing insight from within the university but outside of the program. Internal consultants must: 1. Be faculty members outside program being reviewed; 2. Not be co author or co creator with faculty within the program being reviewed. 3. Be APSU Graduate Faculty (if graduate program is reviewed); Tips for identifying potential reviewers: Ask appropriate professional associations for help in identifying potential reviewers. Many disciplinary organizations provide training for program reviewers and can provide names of experienced/trained individuals. Ask department faculty for suggestions. 7 P age

Contact comparable programs at other regional universities to learn who successfully reviewed their programs. The Office of Institutional Effectiveness and Assessment pays reviewers and will reimburse program reviewers for travel costs and provide the per diem rate for meals and incidentals. The Office of Institutional Effectiveness and Assessment does not provide a stipend or other compensation for the internal consultants. What is included in the self study? The narrative of the self study should be constructed after an open and frank discussion by program faculty and staff members as they prepare for the review. Individual faculty and/or committees can be appointed to write the self study, but the program chair is responsible for the final product. Use the self study report templates for associate, baccalaureate and graduate programs (Appendices B, C, and D) to organize the self study. While completing the self study, use the following points as a guide: Use a five year timeframe when referencing program history and accomplishments. Avoid naming particular faculty members when citing examples. Contact Information Technology, Distance Education, Library, Finance and Administration, Decision Support and Institutional Research and Enrollment Management offices for resource assistance. When preparing the listing of program faculty, include faculty who may only teach in your program once every year or two. Specify what courses faculty teach and how often. What additional information might the reviewers need? If any of the data requested in the outline above is too cumbersome or lengthy to include in full in an appendix, then summarize and include either a) a website where the exhibits are posted; or, b) a note that the exhibit will be available at the site visit. Physical exhibits should be gathered in a convenient location in case they are requested by the review team at the time of the site visit. Among those materials that could be available: Written exams, reports, projects, etc. used for Institutional Effectiveness over the past five years; Previous Academic Audit or program review narrative report and summary document; Syllabi for all courses in the program; Journal articles from students or student/faculty collaborations; Research presentations from students. How do the reviewers assess the program? The review team reads the self study and related materials before the visit and notes questions and concerns to be addressed during the review. During the site visit, they observe, question, and assess the program in light of the self study. They may also examine additional information that you will prepare for their perusal. 8 P age

What are the responsibilities of the program reviewer(s)? Read the program s self study and the applicable THEC program review rubric before arriving on campus. Conduct interviews of faculty, administrators, students, and employers/internship supervisors. Before adjourning, the program reviewers complete the THEC checklist for the designated program level (see Appendices E, F, and G). (When writing the self study, address every checklist point included in the rubric. Using this outline will make it easier for reviewers and ensure compliance with performance funding guidelines.) Provide a verbal report to the faculty, staff and administrators before leaving campus. Prepare and submit a narrative report to the Director of Institutional Effectiveness and Assessment within 30 calendar days after the visit. How is the site visit scheduled? After reviewers are selected and approved, the Quality Assurance Coordinator will correspond with them to confirm their participation and send them all of the necessary materials regarding the self study, including reviewer guidelines, university bulletins, and self study documents. Once the reviewers arrive in Clarksville, the academic department will act as their host and therefore be responsible for transportation and information. The faculty and staff of the program under review are responsible for: Scheduling rooms for departmental sessions (see Appendix A for details) and collaborating with Quality Assurance Coordinator to create complete schedule; Schedule participation of departmental faculty, students, and stakeholders; Distributing schedule to departmental participants (Quality Assurance Coordinator will distribute to administrators and reviewers); Arrange meal events and refreshments; Provide local transportation for review team. 9 P age

APPENDIX A Sample Draft Email Invitation for External Reviewers Dear [Potential Reviewer], Austin Peay State University s [name of program to be reviewed] will undergo a Program Review evaluation this academic year as part of the Tennessee Higher Education Commission s Quality Assurance Funding process. This program review is conducted once every five years in an effort to help our program improve the quality of the educational experience we offer students. This review includes an evaluation of a comprehensive self study report as well as an on site visit by a team of three reviewers (one external reviewer and two internal reviewers) that will occur next spring between mid March and mid April. To prepare for this process, [name of program to be reviewed] has been asked to identify an external reviewer from outside Tennessee to assist two internal reviewers that will be chosen from other disciplines within the university. This distinguished scholar, external to Austin Peay State University, will lead the team in analyzing our program; interviewing faculty, students, employers, and administration; compiling a written summary report and completing standardized evaluation forms. We would be honored if you would consider serving in this role. Our program and Austin Peay would greatly appreciate your participation. We will be glad to coordinate this visit with your schedule in mind and provide a stipend and reimburse for authorized travel expenses. Additional information about the Program Review process is available for your information on APSU s website at http://www.apsu.edu/dsir/prresources.php. Can you please consider this opportunity and inform us of your decision by [date]? We will need a copy of your current vita or a link to a web based version for review. Once again, thank you for your thoughtful consideration of the commitment to this process. Sincerely, [Name and Title of Chair] [Contact Information of Chair] 10 P age

APPENDIX B Itinerary of Program Review Visit Schedule Academic Department Name Degree and Title of Program Date of Visit External Program Reviewer(s) Name, Title, Institution Name, Title, Institution Internal Program Reviewers Name, Title, Department Name, Title, Department Day 1 Date (Academic Department schedules travel to and from hotel.) 7:30 a.m. Pick up from hotel Program Affiliate 8:00 a.m. THEC Quality Assurance Funding Program Review Orientation and Breakfast meeting (Institutional Effectiveness and Assessment schedules location and attendees.) Review Team (External and Internal) Dean of the College of Graduate Studies Vice Provost and Associate Vice President for Academic Affairs Director of Institutional Effectiveness and Assessment Quality Assurance Coordinator 9:00 a.m. Tour and Overview of Department (Academic Department schedules location.) Review Team (External and Internal) Chair of Department 9:45 10:00 a.m. Break 10:00 a.m. Faculty Interviews (Academic Department schedules location and attendees.) Review Team (External and Internal) Faculty 11:00 11:50 a.m. Interview with Dean (Academic Department schedules location.) Review Team (External and Internal) Dean of College (and Dean of the College of Graduate Studies, if applicable) 11 P age

12:00 p.m. Luncheon Meeting (Academic Department schedules location, attendees, and transportation.) Review Team (External and Internal) Chair of Department Available Faculty Employers and Internship Supervisors 1:15 p.m. Interview with Students (Academic Department schedules location.) Review Team (External and Internal) Students (majors only) 2:00 3:00 p.m. Administration (Institutional Effectiveness and Assessment schedules location and attendees.) Review Team (External and Internal) Provost and Vice President for Academic Affairs Vice Provost and Associate Vice President for Academic Affairs Executive Director for Decision Support and Institutional Research Director of Institutional Effectiveness and Assessment Director of Library 3:00 p.m. Review Team Work Session (Academic Department schedules location.) Reviewers begin outline of summary report. External Reviewers complete forms: THEC Appendix E: Program Review: Certificate and Associate Programs THEC Appendix F: Program Review: Baccalaureate Programs THEC Appendix G: Program Review: Graduate Programs 4:00 p.m. Concluding Session (Institutional Effectiveness and Assessment schedules location and attendees.) (Evaluation forms are collected from program reviewers.) Review Team (External and Internal) Chair of Department Dean of College Available Department Faculty Provost and Vice President of Academic Affairs Vice Provost and Associate Vice President for Academic Affairs Dean of College of Graduate Studies (Graduate programs only) Director of Institutional Effectiveness and Assessment Quality Assurance Coordinator (Addresses and Phone Numbers for Reference) Chair of Department Office of Institutional Effectiveness and Assessment 12 P age

Appendix C Associate Program Review Self-Study Report Department of Department Name Austin Peay State University Self-Study Report for Degree in Program Name Program Review Audit Year Program Review Team Lead: Name Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s) Focal Area 7: Name(s) 13 P age

I. INTRODUCTION Department or Program Mission: Program History and Structure: Introduce the program. Describe program structure such as college and department program is housed, if program offered partly or entirely online, other special characteristics of program. Include a brief history if applicable to understanding of program s current status. Faculty: Full Time Part Time Full Professor Associate Professor Assistant Professor Instructor Adjunct Figure 1. Head count of program name current faculty. Other faculty and staff assigned administrative duties, etc. Student Demographics: Minority Non-Minority Total Freshmen Sophomore Junior Senior Total Figure 2. Head count in program name fall 2018. Describe appropriate unique characteristics of program students. Program Review Process Describe how the program conducted its self-study process who was engaged (faculty, adjunct faculty, stakeholders, students); how they were engaged (meetings, online methods, focus groups, survey, etc.); and how the self-study report was drafted, reviewed, and finalized. 14 P age

II. OVERALL PERFORMANCE This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of curriculum, student experience and faculty. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the Program Review self-study processes benefited the program should be included in this section. III. FOCAL AREAS Focal Area 1: Learning Outcomes Program learning outcomes: 1. 2. 3. 4. 5. 6. 7. 8. Student learning outcomes of core courses: Course #1 1. 2. 3. 4. Course #2 1. 2. 3. Etc. Describe the process for evaluating program and student learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field. Reviewers will be looking to identify the following information: Program and student learning outcomes are clearly identified and measurable. Program uses appropriate indicators to evaluate achievement of program and student learning outcomes. Program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement. Program directly aligns with the institution s mission. 15 P age

Focal Area 2: Curriculum Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum content and organization. How often does this occur, and who is involved in this process? Give examples. Describe the process of ensuring courses are offered regularly and that students can make timely progress towards their degree. Give examples. Describe the process of how the faculty incorporates appropriate pedagogical and/or technological innovations into the curriculum that enhance student learning. Give examples. Describe how the curriculum is aligned with and contributes to the mastery of program and student learning outcomes identified in Focal Area 1. Give examples. Describe how the curricular content of the program reflects current standards, practices, and issues in the disciplines. Give examples. Describes how the curriculum fosters analytical and critical thinking and problem-solving. Give examples. Describe how the design of the program s specific courses provides students with a solid educational foundation. Give examples. Describe and explain how the curriculum is appropriate to the level and purpose of the program. Give examples. Focal Area 3: Student Experience Describe how the program provides students with opportunities to apply what they have learned to situations outside the classroom. Give examples. Describe how the program provides students with the opportunity to regularly evaluate faculty relative to the quality of their teaching effectiveness. Describe how the program ensures students are exposed to professional and career opportunities appropriate to the field of study. Describe how students in the program have access to appropriate academic support services. Focal Area 4: Faculty (Full-time and Part-time) Describe how all faculty (full-time and part-time) meet high standards set by the program and expected SACSCOC guidelines for credentials. Describe how the number of faculty in the program are able to meet the needs of the program with appropriate teaching loads. Describe how faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline. Describe how the program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. Give examples. Describe how faculty engages in regular professional development that enhances their teaching, scholarship and practice. 16 P age

Give examples. Describe how faculty actively engage in planning, evaluation and improvement processes that measure and advance student success. Give examples. Focal Area 5: Learning Resources Discuss how library, equipment, and facilities are regularly evaluated, encouraging necessary improvements within the context of overall institutional resources. Discuss how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples. Focal Area 6: Economic Development For transfer programs: Describe how the program provides and promotes clear transfer pathways supported by curricular maps, advising, and other means to support student articulation. Gives examples. Describe the success of graduates who pursue baccalaureate degrees in related programs. Give statistics. For career programs: Describe how the program demonstrates responsiveness to local and regional workforce needs through an advisory committee, partnerships with industry and/or other means. Give examples. Describe how the program identifies applicable workforce trends and uses the information to improve the program. Give examples. Focal Area 7: Support Demonstrate how the program s operating budget is consistent with the needs of the program. Provide historical program enrollment and degree awards to demonstrate they are sufficient to sustain high quality and costeffectiveness. 250 200 150 100 50 English BA/BS Fall Census Enrollment 0 2011 2013 2014 2015 2016 Enrollment Figure 3. Head count of program name fall enrollments between 2011 and 2016. 17 P age

English BA/BS Degree Awards 60 50 40 30 20 10 0 2011 12 2012 13 2013 14 2014 15 2015 16 Degree Awards Figure 4. Number of degrees in program name awarded per year between 2011 and 2016. IV. APPENDICES Appendix 1: Appendix 2: Appendix 3: Appendix 4: Appendix 5: A Table of Contents for the Appendices is needed. 18 P age

Appendix D Baccalaureate Program Review Self-Study Report Department of Department Name Austin Peay State University Self-Study Report for Degree in Program Name Program Review Audit Year Program Review Team Lead: Name Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s) 19 P age

I. INTRODUCTION Department or Program Mission: Program History and Structure: Introduce the program. Describe program structure such as college and department program is housed, if program offered partly or entirely online, other special characteristics of program. Include a brief history if applicable to understanding of program s current status. Faculty: Full Time Part Time Full Professor Associate Professor Assistant Professor Instructor Adjunct Figure 1. Head count of program name current faculty. Other faculty and staff assigned administrative duties, etc. Student Demographics: Minority Non-Minority Total Freshmen Sophomore Junior Senior Total Figure 2. Head count in program name fall 2018. Describe appropriate unique characteristics of program students. Program Review Process Describe how the program conducted its self-study process who was engaged (faculty, adjunct faculty, stakeholders, students); how they were engaged (meetings, online methods, focus groups, survey, etc.); and how the self-study report was drafted, reviewed, and finalized. II. OVERALL PERFORMANCE This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of curriculum, student experience and faculty. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the Program Review self-study processes benefited the program should be included in this section. 20 P age

III. FOCAL AREAS Focal Area 1: Learning Outcomes Program learning outcomes: 1. 2. 3. 4. 5. 6. 7. 8. Student learning outcomes of core courses: Course #1 1. 2. 3. 4. Course #2 1. 2. 3. Etc. Describe the process for evaluating program and student learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field. Reviewers will be looking to identify the following information: Program and student learning outcomes are clearly identified and measurable. Program uses appropriate indicators to evaluate achievement of program and student learning outcomes. Program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement. Program directly aligns with the institution s mission. Focal Area 2: Curriculum Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum. How often does this occur, and who is involved in this process? Give examples. Describe how the program has developed a process to ensure courses are offered regularly and that students can make timely progress towards their degree. Give examples. Describe how the program incorporates appropriate pedagogical and/or technological innovations that enhance student learning in the curriculum. Give examples. Describe how the curriculum is aligned with and contributes to the mastery of program and student learning outcomes identified in Focal Area 1. Give examples. Describe how the curricular content of the program reflects current standards, practices, and issues in the discipline. Give examples. 21 P age

Describes how the curriculum fosters analytical and critical thinking and problem-solving. Give examples. Describe how the design of the program s specific courses provides students with a solid educational foundation. Give examples. Describe how the curriculum reflects a progressive challenge to students and that depth and rigor effectively prepares students for careers or advanced study. Describe how the curriculum encourages the development of and the presentation of results and ideas effectively and clearly in both written and oral discourse. Give examples. Describe how the curriculum exposes students to discipline-specific research strategies from the program area. Focal Area 3: Student Experience Describe how the program provides students with opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness. Describe how the program ensures students are exposed to professional and career opportunities appropriate to the field of study. Describe how the program provides students with the opportunity to apply what they have learned to situations outside the classroom. Describe how the program seeks to include diverse perspectives and experiences through curricular and extracurricular activities. Describe how students in the program have access to appropriate academic support services. Focal Area 4: Faculty (Full-time and Part-time) Describe how all faculty (full-time and part-time) meet high standards set by the program and expected SACSCOC guidelines for credentials. Describe how the number of faculty in the program are able to meet the needs of the program with appropriate teaching loads. Describe how faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline. Describe how the program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. Give examples. Describe how faculty engages in regular professional development that enhances their teaching, scholarship and practice. Describe how faculty actively engage in planning, evaluation and improvement processes that measure and advance student success. Give examples. Focal Area 5: Learning Resources Discuss how library, equipment, and facilities are regularly evaluated, encouraging necessary improvements within the context of overall institutional resources. Discuss how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples. 22 P age

Focal Area 6: Support Demonstrate how the program s operating budget is consistent with the needs of the program. Provide historical program enrollment and degree awards to demonstrate they are sufficient to sustain high quality and costeffectiveness. 250 200 150 100 50 English BA/BS Fall Census Enrollment 0 2011 2013 2014 2015 2016 Enrollment Figure 3. Head count of program name fall enrollments between 2011 and 2016. 60 50 40 30 20 10 English BA/BS Degree Awards 0 2011 12 2012 13 2013 14 2014 15 2015 16 Degree Awards Figure 4. Number of degrees in program name awarded per year between 2011 and 2016. Describe how the program is responsive to local, state, regional and national needs. Give examples. 23 P age

IV. APPENDICES Appendix 1: Appendix 2: Appendix 3: Appendix 4: Appendix 5: A Table of Contents for the Appendices is needed. 24 P age

Appendix E Graduate Program Review Self-Study Report Department of Department Name Austin Peay State University Self-Study Report for Degree in Program Name Program Review Audit Year Program Review Team Lead: Name Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s) 25 P age

I. INTRODUCTION Department or Program Mission: Program History and Structure: Introduce the program. Describe program structure such as college and department program is housed, if program offered partly or entirely online, other special characteristics of program. Include a brief history if applicable to understanding of program s current status. Faculty: Full Time Part Time Full Professor Associate Professor Assistant Professor Instructor Adjunct Figure 1. Head count of program name current faculty. Other faculty and staff assigned administrative duties, etc. Student Demographics: Minority Non-Minority Total First Year Second Year More than 2 Years Total Figure 2. Head count in program name fall 2018. Describe appropriate unique characteristics of program students. Program Review Process Describe how the program conducted its self-study process who was engaged (faculty, adjunct faculty, stakeholders, students); how they were engaged (meetings, online methods, focus groups, survey, etc.); and how the self-study report was drafted, reviewed, and finalized. II. OVERALL PERFORMANCE This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of teaching and student learning. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the program review audit self-study processes benefited the program should be included in this section. 26 P age

III. FOCAL AREAS Focal Area 1: Learning Outcomes Program learning outcomes: 1. 2. 3. 4. 5. 6. 7. 8. Student learning outcomes of core courses: Course #1 1. 2. 3. 4. Course #2 1. 2. 3. Etc. Describe the process for evaluating program and course-level learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field. Reviewers will be looking for the following information: Program and student learning outcomes are clearly identified and measurable. Program uses appropriate evidence to evaluate achievement of program and student learning outcomes. Program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement. Program directly aligns with the institution s mission. Focal Area 2: Curriculum Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum. How often does this occur, and who is involved in this process? Give examples. Describe the process developed to ensure courses are offered regularly and that students can make timely progress toward their degree. Give examples. Describe how the program reflects progressively more advanced in academic content than its related undergraduate program(s). Give examples. Describe how the curriculum is aligned with and contributes to mastery of program and student learning outcomes identified in Focal Area 1. Give examples. Describe how the curriculum is structured to include knowledge of the literature of the discipline. Give examples. 27 P age

Describe how the curriculum strives to offer ongoing student engagement in research and/or appropriate professional practice and training experiences. Give examples. If the program is offered entirely online, describe how the program is evaluated regularly to assure achievement of program outcomes are equivalent to on-campus programs. Give examples if applicable. Describe how the program incorporates appropriate pedagogical and/or technological innovations into the curriculum that advance student learning. Give examples. Focal Area 3: Student Experience Describe how the program ensures a critical mass of students to ensure an appropriate group of their peers participating in course work. Give examples. Describe and explain how the program provides students with the opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness. Describe how the program provides adequate professional development opportunities, such as encouraging membership in professional associations, participation in conferences and workshops, and opportunities for publication. Give examples. Describe how the program provides adequate enrichment opportunities, such as lecture series, to promote a scholarly environment. Give examples. Describe how the program seeks to include diverse perspectives and experiences through curricular and extracurricular activities. Give examples. Describe and explain how students have access to appropriate academic support services. Focal Area 4: Faculty Describe how all faculty, full-time and part-time, meet the high standards set by the program and expected SACSCOC guidelines for credentials. Describe and explain how faculty teaching loads are aligned with the highly individualized nature of graduate instruction, especially the direction of theses or dissertations. Describe how faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline. Explain how faculty engages in regular professional development that enhances their teaching, scholarship and practice. Describe how faculty is actively engaged in planning, evaluation and improvement processes that measure and advance student success. Demonstrate how the faculty uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. This could include analysis of course evaluations, peer observations, SLO assessments, etc. How often does the faculty do this? Focal Area 5: Learning Resources Describe how the program regularly evaluates its equipment and facilities, encouraging necessary improvements within the context of overall institutional resources. Explain how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples. Explain how the program provides adequate materials and support staff to encourage research and publication. Give examples. 28 P age

Focal Area 6: Support Demonstrate how the program s operating budget is consistent with the needs of the program. Provide historical program enrollment and degree awards to demonstrate they are sufficient to sustain high quality and costeffectiveness. 250 200 150 100 50 English BA/BS Fall Census Enrollment 0 2011 2013 2014 2015 2016 Enrollment Figure 3. Head count of program name fall enrollments between 2011 and 2016. 60 50 40 30 20 10 English BA/BS Degree Awards 0 2011 12 2012 13 2013 14 2014 15 2015 16 Degree Awards Figure 4. Number of degrees in program name awarded per year between 2011 and 2016. 29 P age

Describe and explain how the program is responsive to local, state, regional, and national needs. Describe how the program regularly and systemically collects data on graduating students and evaluates placement of graduates. Give examples. Describe how the program s procedures are regularly reviewed to ensure alignment to institutional policies and mission. Give examples. IV. APPENDICES Appendix 1: Appendix 2: Appendix 3: Appendix 4: Appendix 5: A Table of Contents for the Appendices is needed. 30 P age

Appendix F 2015-20 Quality Assurance Funding Program Review: Certificate and Associate Programs Institution: Program Title: CIP Code: Embedded Certificates: Embedded Certificates: Embedded Certificates: Instruction for External Reviewer(s) In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable certificate and associate program undergoes either an academic audit or external peer review according to a pre-approved review cycle. If the program under review contains embedded Technical Certificates, the names of each certificate should be included above. The review of embedded certificates must be included as part of the review of the program in which they are embedded. Embedded certificates do not require a separate Program Review Rubric. The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 30 criteria grouped into seven categories. THEC will use these criteria to assess standards and distribute points to certificate and associate programs. The five criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment. For each criterion within a standard, the responsible program has provided evidence in the form of a Self-Study. Supporting documents will be available for review as specified in the Self-Study. As the external reviewer, you should evaluate this evidence and any other evidence observed during the site visit to determine whether each criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is inappropriate or not applicable to the program under review, the item should be marked NA. This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the department, college and central administration, as well as the Tennessee Higher Education Commission. When combined with the written report, prepared by the entire program review committee, the Program Review Rubric will facilitate development of a program action plan to ensure continuous quality improvement. Your judgment of the criteria will be used in allocating state funds for the community college's budget. Name, Title and Institutional Affiliation of Reviewer(s) Name Title Institution Signature Date Name Title Institution Signature Date 31 P age

Program Review Rubric Certificate and Associate Programs Directions: Please rate the quality of the academic program by placing a checkmark in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent evidence of meeting the criterion. 1. Learning Outcomes N/A Poor Fair Good Excellent 1.1 Program and student learning outcomes are clearly identified and measurable. 1.2 The program uses appropriate indicators to evaluate achievement of program and student learning outcomes. 1.3 The program makes uses of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement. 1.4 The program directly aligns with the institution's mission. 2. Curriculum N/A Poor Fair Good Excellent 2.1 The curriculum content and organization are reviewed regularly and the results are used for curriculum improvement. 2.2 The program has developed a process to ensure courses are offered regularly and that students can make timely progress towards their degree. 2.3 The program incorporates appropriate pedagogical and/or technological innovations that enhance student learning into the curriculum. 2.4 The curriculum is aligned with and contributes to mastery of program and student learning outcomes identified in 1.1. 2.5 The curricular content of the program reflects current standards, practices, and issues in the discipline. 2.6 The curriculum fosters analytical and critical thinking and problem-solving. 2.7 The design of degree program specific courses provides students with a solid foundation. 2.8 The curriculum is appropriate to the level and purpose of the program. 3. Student Experience N/A Poor Fair Good Excellent 3.1 The program provides students with the opportunity to apply what they have learned to situations outside the classroom. 3.2 The program provides students with the opportunity to regularly evaluate faculty relative to the quality of their teaching effectiveness. 3.3 The program ensures students are exposed to professional and career opportunities appropriate to the field. 3.4 Students have access to appropriate academic support services. 4. Faculty (Full-time and Part-time) N/A Poor Fair Good Excellent 32 P age

4.1 All faculty, full time and part-time, meet the high standards set by the program and expected SACSCOC guidelines for credentials. 4.2 The faculty are adequate in number to meet the needs of the program with appropriate teaching loads. 4.3* The faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline. 4.3 The program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. 4.4 The faculty engage in regular professional development that enhances their teaching, scholarship, and practice. 4.5 The faculty are actively engaged in planning, evaluation and improvement processes that measure and advance student success. 5. Learning Resources N/A Poor Fair Good Excellent 5.1 * The program regularly evaluates its equipment and facilities, encouraging necessary improvements within the context of overall institutional resources. 5.2 The program has access to learning and information resources that are appropriate to support teaching and learning. 6. Economic Development N/A Poor Fair Good Excellent 6.1 For transfer programs: The program provides and promotes clear transfer pathways supported by curricular maps, advising and other means to support student articulation. 6.2 * For transfer programs: Graduates who transfer to baccalaureate programs in a related area are successful. 6.3 For career programs: The program demonstrates responsiveness to local and regional workforce needs through an advisory committee, partnerships with industry and/or other means. 6.4 For career programs: The program identifies applicable workforce trends and uses the information to improve the program. 7. Support N/A Poor Fair Good Excellent 7.1 * The program's operating budget is consistent with the needs of the program. 7.2 * The program has a history of enrollment and/or graduation rates sufficient to sustain high quality and cost-effectiveness. *Criteria not scored as part of Quality Assurance Funding. 33 P age

Appendix G 2015-20 Quality Assurance Funding Program Review: Baccalaureate Programs Institution: Program Title: CIP Code: Instruction for External Reviewer(s) In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable baccalaureate program undergoes either an academic audit or external peer review according to a pre-approved review cycle. The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 30 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute points in to baccalaureate programs. The four criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment. For each criterion within a standard, the responsible program has provided evidence in the form of a Self-Study. Supporting documents will be available for review as specified in the Self-Study. As the external reviewer, you should evaluate this evidence and any other evidence observed during the site visit to determine whether each criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is inappropriate or not applicable to the program under review, the item should be marked NA. This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the department, college and central administration, as well as the Tennessee Higher Education Commission. When combined with the written report, prepared by the entire program review committee, the Program Review Rubric will facilitate development of a program action plan to ensure continuous quality improvement. Your judgment of the criteria will be used in allocating state funds for the university's budget. Name Title Institution Signature Date Name, Title and Institutional Affiliation of Reviewer(s) Name Title Institution Signature Date 34 P age

Program Review Rubric Baccalaureate Programs Directions: Please rate the quality of the academic program by placing a checkmark in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent evidence of meeting the criterion. 1. Learning Outcomes N/A Poor Fair Good Excellent 1.1 Program and student learning outcomes are clearly identified and measurable. 1.2 The program uses appropriate evidence to evaluate achievement of program and student learning outcomes. 1.3 The program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement. 1.4 The program directly aligns with the institution's mission. 2. Curriculum N/A Poor Fair Good Excellent 2.1 The curriculum content and organization are reviewed regularly and results are used for curricular improvement. 2.2 The program has developed a process to ensure courses are offered regularly and that students can make timely progress towards their degree. 2.3 The program incorporates appropriate pedagogical and/or technological innovations that enhance student learning into the curriculum. 2.4 The curriculum is aligned with and contributes to mastery of program and student learning outcomes identified in 1.1. 2.5 The curricular content of the program reflects current standards, practices, and issues in the discipline. 2.6 The curriculum fosters analytical and critical thinking and problem-solving. 2.7 The design of degree program specific courses provides students with a solid foundation. 2.8 The curriculum reflects a progressive challenge to students and that depth and rigor effectively prepares students for careers or advanced study. 2.9 The curriculum encourages the development of and the presentation of results and ideas effectively and clearly in both written and oral discourse. 2.10 The curriculum exposes students to discipline-specific research strategies from the program area. 3. Student Experience N/A Poor Fair Good Excellent 3.1 The program provides students with opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness. 3.2 The program ensures students are exposed to professional and career opportunities appropriate to the field. 3.3 The program provides students with the opportunity to apply what they have learned to situations outside the classroom. 3.4 The program seeks to include diverse perspectives and experiences through curricular and extracurricular activities. 35 P age

3.5 Students have access to appropriate academic support services. 4. Faculty (Full-time and Part-time) N/A Poor Fair Good Excellent 4.1 All faculty, full time and part-time, meet the high standards set by the program and expected SACSCOC guidelines for credentials. 4.2 The faculty are adequate in number to meet the needs of the program with appropriate teaching loads. 4.3* The faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline. 4.4 The program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. 4.5 The faculty engages in regular professional development that enhances their teaching, scholarship and practice. 4.6 The faculty is actively engaged in planning, evaluation and improvement processes that measure and advance student success. 5. Learning Resources N/A Poor Fair Good Excellent 5.1* The program regularly evaluates its equipment and facilities, encouraging necessary improvements within the context of overall institutional resources. 5.2 The program has access to learning and information resources that are appropriate to support teaching and learning. 6. Support N/A Poor Fair Good Excellent 6.1* The program's operating budget is consistent with the needs of the program. 6.2* The program has a history of enrollment and/or graduation rates sufficient to sustain high quality and cost-effectiveness. 6.3 The program is responsive to local, state, regional, and national needs. *Criteria not scored as part of Quality Assurance Funding. 36 P age

Appendix H 2015-20 Quality Assurance Funding Program Review: Graduate Programs Institution: Program Title: CIP Code: Degree Designation: Instruction for External Reviewer(s) In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable graduate program undergoes either an academic audit or external peer review according to a pre-approved review cycle. The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 32 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute points in to graduate programs. The four criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment. For each criterion within a standard, the responsible program has provided evidence in the form of a Self-Study. Supporting documents will be available for review as specified in the Self-Study. As the external reviewer, you should evaluate this evidence and any other evidence observed during the site visit to determine whether each criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is inappropriate or not applicable to the program under review, the item should be marked NA. This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the department, college and central administration, as well as the Tennessee Higher Education Commission. When combined with the written report, prepared by the entire program review committee, the Program Review Rubric will facilitate development of a program action plan to ensure continuous quality improvement. Your judgment of the criteria will be used in allocating state funds for the university's budget. Name, Title and Institutional Affiliation of Reviewer(s) Name Title Institution Signature Date Name Title Institution Signature Date 37 P age