Framework for Program Review

Transcription

1 Framework for Program Review Approving authority Director, Learning Futures Approval date 5 December 2017 Advisor Next scheduled review 2023 Dr Jude Williams Program Teaching Quality, Learning Futures jude.williams@griffith.edu.au (07) Document URL for Program Review.pdf TRIM document 2017/ Description The Framework for Program Review (Framework) is designed to support review panel stakeholders as y discuss ir experiences of program gar analyse data to inform ir decisions about its future. Related documents Program Approval Review Program Review Template Graduate Research Good Practice Principles The Framework for Program Review (Framework) is designed to support review panel stakeholders as y discuss ir experiences of program gar analyse data to inform ir decisions about its future. The Framework lists terms of reference that are typical of those written by review panels at start of review process. As terms of reference are generally broad, Framework also outlines aspects of a program (related to each term of reference) that may be ed when evaluating how well program is performing meeting regulatory requirements. These aspects are referred to as criteria. It is anticipated that each review panel may add to or amend terms of reference list of criteria to reflect specific circumstances of a particular program, for example, in programs that have specific professional accreditation requirements or characteristics that may change focus of review. The criteria have been written with Higher Education Stards Framework (2015) in mind show how review panel might use data to demonstrate program s compliance with national regulatory stards. An overview of relationship between higher education stards (italics) Framework s criteria is as follows: Stard 3.1 Course Design Continued relevance fit for purpose Currency appropriateness of program content structure Annual Monitoring Stard 3.2 Staffing, Stard 3.3 Learning Resources Educational Support, & Stard 4.2 Research Training Viability sustainability Learning environment (including physical resources) Stard 1.1 Admission Student participation achievement (enrolments, progression, retention, outcomes, employment) Stard 1.4 Learning Outcomes Assessment Learning outcomes student achievement 1 Framework for Program Review

2 Quality of assessment Program Leadership Staff quality/characteristics Quality of teaching Stard 5.3 Monitoring, Review Improvement Relevance effectiveness of program for stakeholders Results of internal or external benchmarking activities Or issues of particular relevance to a program The first column of Framework contains typical terms of reference that have been developed used by previous review panels at Griffith University. The second column provides list of criteria a series of questions designed to prompt discussions among stakeholders about how well program is performing. These questions are not a checklist. In third column re appear, where appropriate, a list of evidence that may be useful in discussions about program performance. This list is not exhaustive review panels may y need to gar additional information from a broader range of sources The fourth column of Framework indicates where information is available or who is responsible for providing information. The cells in fourth column that are shaded grey indicate that data will be automatically generated by OPS reported as pre-populated data in a program review portfolio. Scope of Program Review What are Questions What might be useful to know? Evidence What evidence could be used? Source/ Responsibility Where is evidence available? Terms of Reference? Program design is appropriate consistent with relevant external professional accreditation bodies Australian Qualifications Framework History of program Assess program s continued relevance fit for purpose What was program rationale? What was market need fit to Group Program Profile? The original program proposal (Part 1 Part 2, section 1). This is available in Corporate Archives Digitisation Services (CRDS) - contact your Academic Services Consultant (ASC) Brief program description What are program level outcomes? From courses program website What are core elective courses? What are capstone courses? Is re an academic plan? What is research component of program? What are possible pathways that students can take to progress to qualification? 2 Framework for Program Review

3 AQF Level What AQF level does program fit? Annual monitoring What process is used to review implement scheduled course program monitoring? What changes have been made to program since inception/last review? What were reasons for se changes? Has re been a tracking or evaluation of impact of se changes? What have been impacts of se changes? Is information about program up to date? What was result of last accreditation review/visit (if applicable)? Were re recommendations? Were recommendations enacted? What was outcome? Is program aligned to profession s stated meta values? Australian Qualifications Framework (AQF) Program website Course profiles Documents on minor major changes submitted to Programs Committee. Number, type scope of changes proposed implemented Survey of stakeholders experiences of program Comprehensive program mapping Conference presentation Program award application Shortlists for awards Program Director/ASC Program resourcing information is adequate Student dem Assess viability sustainability of program What is number of current enrolments? What is enrolment trend over 5 years? What are characteristics of students of this program? Student load current over past 5 years Student Profile data (current cohort trend over past 5 years)(gender, age, Socio- Economic Status (SES), ethnicity, educational background) Proportion of load from Queensl/ interstate / international What is dem in market for this program relative to competitors offering like qualifications in same Field of Education? Queensl Tertiary Admissions Centre (QTAC) dem: 1st preferences % of 1st preferences in Field of Education (FOE) (market perception indicator) 6 digit FOE detail (proposed new) 1-6 preference numbers (proposed new) 3 Framework for Program Review

4 What proportion of students expressing an initial interest obtaining an offer go on to enrol? Program Ranking based on preference (provide a rank ordered list to show % of market) (proposed new) Offer acceptance enrolment ratios numbers Staff quality/characteristics Teaching Learning are of high quality Do staff possess appropriate levels of qualifications expertise to teach this program? Do y possess current knowledge of ir discipline have an understing of pedagogical /or adult learning principles? Are academic staff equipped to design deliver online courses? Are staff teaching a postgraduate program actively carrying out research, publishing in relevant discipline area appropriately supervising students in scholarly pursuits research? Compliance risk Does program comply with professional body accreditation requirements? Have risks been identified, ed minimised, i.e. reliance on third parties to deliver program, high reliance on casual teaching staff, high students teaching staff ratio, declining admissions stards, student complaints? Program leadership How do program directors communicate with course convenors, Deans (L&T), Head of School year level coordinators students about aspects of program its design, including moderation procedures ensuring consistent appropriate assessment? Staff Profile Numbers of teaching staff, ir qualifications, expertise, awards grants in Learning Teaching (L&T) Number of teaching staff who are fellows of Higher Education Academy Staff Profile Academic publications Higher Degree (HDR) supervision Staff Profile 4 Framework for Program Review

5 Admission criteria are appropriate Admission Assess program s performance in terms of student participation achievement Are admission criteria for program appropriate, take account of external benchmarks ensure that students have adequate prior knowledge skills to undertake course of study successfully? Orientation transfer What strategies are in place for supporting, inducting connecting with new students to this program? Alignment with Undergraduate Programs Admission Policy. Alignment with Postgraduate Programs Admission Policy Starting@Griffith Preparation orientations; connection engagement; learning pastoral support s Progression retention What is retention rate for program? At what points does student attrition from program occur? What strategies have been used to date to enhance student retention? Overall Retention Retention First Year to Second Year % Retained (new) In Program Retention for target groups (low SES), indigenous etc) At Griffith (trended) Program completion rate Program performance indicators in BI/Management Information Server (MIS) section of planning support site Assessment is effective expected student learning outcomes are achieved Learning outcomes - student achievement What is GPA failure rate of students in each year of program (current for past 5 years)? How does program ensure equivalent student learning outcomes regardless of a student s place or mode of study? (as applicable) Are program level learning outcomes achieved stard of achievement moderated? Are students obtaining generic employability skills as a consequence of ir studies? Are students acquiring Griffith Graduate Attributes? Are students gaining employment after graduating? Innovative Research Universities (IRU) Calibration project data Generic Skills Graduate Destination Survey (GDS) - Graduate Success Griffith Destination graph 5 Framework for Program Review

6 Are y going on to furr study (in like FOE or different FOEs)? Graduate Experience Program monitoring Examine learning environment in which program is offered Are re sufficient resources to meet dem for program? Do students have access to adequate IT, electronic or physical library information sources, specialist equipment in order to achieve program s learning outcomes ensure program quality? Student staff Feedback Employer/Industry feedback Curriculum Consultants Student Feedback What processes are used to ensure cross-campus consistency equivalence of student experience (where applicable)? What impact, if any, are staffstudent ratios having on student experience, learning satisfaction? What strategies are in place to address issues identified? Student:Staff Ratio Student feedback to discuss with Group Blended Learning Advisor (BLA) Evaluate currency approriateness of program content structure Does program: provide students with opportunity to develop knowledge, skills application of knowledge appropriate for its AQF level link ory to practice? ensure that course program level outcomes can be assessed attained? provide a developmental sequence in student s learning? draw from current emerging knowledge in one or more disciplines include relevant oretical frameworks research findings? have gaps or repetition in program content? take into ation entry exit pathways, including articulation from or studies to furr studies? ensure development of Griffith Graduate Attributes? Student feedback Course Experience Questionnaire (CEQ) /Student Experience of Course (SEC) qualitative comments SEC/SET data Mapping from Course Profile system CEQ data comments Survey of stakeholders experiences of program How has program been internationalised? 6 Framework for Program Review

7 Do undergraduate programs provide students with opportunity to complete at least one of following units of study: work integrated learning, service learning, a research project/practicum, or an international study experience? Have appropriate career employment content been embedded in program? Has program been designed so that it can be studied online, on-campus or flexibly with a mix of modes? Evaluate quality of teaching What is quality of teaching across courses of program? Are industry personnel invited to speak/participate in delivery of program as when appropriate? What is level of student satisfaction with teaching quality? Are teaching strategies adopted for delivery of online courses appropriate for this mode of delivery? Does program meet Griffith s good practice stards for online teaching learning? Good Teaching Scale Peer Assisted Course Enhancement (PACE)/Peer Review of Teaching (PROT) data, International Student Barometer (ISB) (teaching) Overall SEC score patterns University Experience Survey (UES) Teaching Quality scale Group BLA Are re processes in place to enable teaching staff to take note of where appropriate act on student feedback of ir teaching? Low SEC score strategy (institution wide) How is technology used in program to enable learning? Group BLA How do learning teaching strategies support students to attain program learning outcomes? Are academic staff reasonably available for students seeking academic assistance? CEQ qualitative comments SEC qualitative comments What arrangements are in place to assure quality of student work placements, practicum or forms of work-integrated learning in course of study, including assuring quality of supervision? Group BLA 7 Framework for Program Review

8 Evaluate quality of assessment Are assessment tasks appropriate for validly assessing attainment of program course level learning outcomes? How are Griffith Graduate Attributes being assessed? Curriculum mapping Review course profile system information Is re an appropriate timing spread of assessment across courses in program? Review course profile system information Are opportunities for formative assessment available? Review course profile system information How is assessment moderated within courses across program? Feedback from Course Convenors, SEC data comments, CEQ comments Is assessment designed conducted by appropriately qualified academic staff, timely, adequate feedback is provided to students on ir assessed work? Evaluate relevance effectiveness of program for stakeholders What are students perceptions of learning outcomes from program (both generic disciplinespecific)? What do major employer stakeholders think about program? What do past graduates think of program? What are views of teaching staff about this program? CEQ qualitative comments Industry advisory group feedback /or Employer feedback / survey GDS, CEQ Alumni feedback survey Teaching staff feedback (request extraction by program) What do or stakeholders think about program? Eg, clinical staff, consumers, professional bodies. Or Terms of Reference 8 Framework for Program Review