AUSTRALIAN UNIVERSITIES QUALITY AGENCY. Report of a Joint Audit of RMIT University

Transcription

1 AUSTRALIAN UNIVERSITIES QUALITY AGENCY Report of a Joint Audit of RMIT University July 2009

2 AUQA Audit Report Number 75 ISBN Australian Universities Quality Agency 2009 Level 10, 123 Lonsdale Street Melbourne, VIC 3000 Ph Fax admin@auqa.edu.au The Australian Universities Quality Agency receives funding from the Australian Government and State and Territory Governments of Australia. The views expressed in this report do not necessarily reflect the views of these governments.

3 CONTENTS 1 EXECUTIVE SUMMARY Audit Findings Main Points Matters from Cycle 1 Audit Theme 1: Internationalisation Theme 2: Industry Engagement National Protocols Other External Reference Points Institutional Context Institutional Profile Strategic Context of the Audit Commendations, Affirmations and Recommendations...6 Commendations... 6 Affirmations... 7 Recommendations MATTERS FROM AUDIT CYCLE Implementation of Cycle 1 Recommendations Follow up: Recommendations on Research and Research Training Quality Systems and Processes Quality Assurance Ensuring Better Alignment Between Planning and Measurement Approach to Academic Standards and Benchmarking THEME: INTERNATIONALISATION Strategic Directions Leadership and Management Global Passport and Internationalisation of the Curriculum Global Passport Internationalisation of Curriculum Student and Staff Mobility Student Mobility Staff Mobility English Language Proficiency Support to International Students RMIT Vietnam Establishment and Governance Staff and Students Infrastructure and IT Program Delivery RMIT as the Asian Hub Offshore Strategy Transnational Education Partnerships Visited by the Panel Delegation...25

4 3.9.1 Bachelor of Communication (Mass Communication) and the Master of Business (Information Technology); Singapore Institute of Management Pte Ltd Bachelor of Applied Science (Information Technology), Taylor s University College, Malaysia Bachelor of Business (International Business), Shanghai Institute of Foreign Trade Bachelor of Engineering (Electrical Engineering), Vocational Training Council of Hong Kong Overall Impressions Based on the Programs Visited Management of Partnerships Standards and Outcome Measures Standards Outcome Measures THEME: INDUSTRY ENGAGEMENT The Industry Engagement Strategy Work Relevant Education Industry in Teaching and Learning Program Development Program Delivery Recognition of Prior Learning Industry Research Networking with the Industry Outcome Measures DATA...38 APPENDICES...49 APPENDIX A1: THE AUDIT PROCESS...49 APPENDIX A2: AQTF AUDIT OF RMIT APPENDIX B: AUQA S MISSION, OBJECTIVES, VISION AND VALUES...60 APPENDIX C: THE AUDIT PANEL...62 APPENDIX D: ABBREVIATIONS AND DEFINITIONS...63

5 RMIT University Audit Report 1 EXECUTIVE SUMMARY This Audit covered the whole University, both higher education and vocational education and training (VET), and was conducted jointly by the Australian Universities Quality Agency (AUQA) and the Victorian Registration and Qualifications Authority (VRQA). This was the first whole of institution audit of a dual sector university. Along with the AUQA requirements for Cycle 2 audit, the audit addressed the VRQA requirements for Australian Quality Training Framework (AQTF) 2007 Essential Standards for Registration. RMIT provided a Portfolio and supporting documents, which addressed matters of relevance to both AUQA and the VRQA. (Further comments provided in section ) 1.1 Audit Findings The scope for the 2009 AUQA audit of RMIT University (hereinafter referred to as RMIT or the University ) is the two themes of Industry Engagement and Internationalisation, together with follow up of selected recommendations from the 2003 AUQA audit. This Report includes comments on other significant matters, including the University s compliance with the National Protocols for Higher Education Approval Processes. The scope for VRQA s audit is the three Essential Standards for Registration that focus on the quality of services and outcomes being achieved for clients as well as Condition of Registration 6. AUQA s detailed findings on the University s actions towards quality assurance and improvement of its standards and outcomes are contained in sections 2 to 4. Data that support the findings are provided in section 5. Information on the conduct of the audit is at Appendix A1. The summary of findings of the VRQA AQTF audit is at Appendix A Main Points RMIT has addressed the recommendations of the 2003 AUQA audit in a very structured way. Strategies that relate to the recommendations have been incorporated into the University s hierarchy of plans and most of the actions identified have been substantially implemented, with continuing implementation and monitoring as appropriate. Research and research training is an area where improvement plans are still under development. Befitting its goals and objectives, RMIT values global engagement of its staff and students, has established global networks of strategic partnerships which support student and staff mobility, and prepares the students well for the new, globalised workplace. RMIT provides a range of effective support services to international students. The ongoing development of the Vietnam campus has the potential to further enhance staff and student mobility. Monitoring of transnational education (TNE) partnerships needs strengthening. RMIT s Industry Engagement Strategy is strongly embedded in teaching and learning, and research in the higher education programs. It has added significant value to the University s engagement with industry and also supports the University in acting on its strategic priorities. It is work in progress and the developments are in the right direction. Work Integrated Learning (WIL), one of the major initiatives of the Strategy, has received explicit attention in the higher education programs and more engagement with it is needed by staff in the technical and further education (TAFE) programs. Industry engagement features prominently within the hierarchy of mainstream plans and related strategies are clearly pivotal to the University s future direction. Australian Universities Quality Agency

6 Audit Report RMIT University With reference to the National Protocols and other selected external reference points, AUQA has not identified any issues of concern. Benchmarking and standards feature as comparators throughout Matters from Cycle 1 Audit In response to 2003 AUQA audit recommendations, RMIT has made major changes to its governance and management processes and systems in a structured way. In particular, changes to the planning, resource allocation and review systems are effective; communication with staff, and staff morale have improved significantly. This Report has commendations on these significant developments. RMIT has a sound planning framework which can be augmented further by ensuring a better alignment between the hierarchy of measures and the hierarchy of plans. Staff should be encouraged further to monitor continuous improvement of quality in a broad range of measures. Research and research training is an area where improvement plans are still under development. RMIT has made significant progress towards clarifying the overall strategy for research and research training. Establishment of high level research institutes has been effective but the University is still working on facilitating commercialisation of research, consolidation of support for graduate students, and the research strategy for its Vietnam campus Theme 1: Internationalisation RMIT has introduced the concept of a Global Passport to encourage internationalisation in all aspects of teaching and learning, and research and the concept is working well to capture some key dimensions of internationalisation. AUQA found some good examples of internationalised curriculum which should be shared across schools. RMIT provides a range of general and dedicated support services to international students. The peer support provided to international students is effective. The progress rates for international students are in general strong and the attrition rates have been decreasing. The development of the Vietnam campus as part of a dual hub approach is working well for RMIT. While the full potential of Vietnam as the Asian Hub is yet to be realised, there is evidence that this has the potential to facilitate two way mobility between Vietnam and Melbourne and between the other Asian partners. This Report commends RMIT on these positive developments and affirms the plans of the University for further enhancement. Based on the five higher education programs visited offshore, AUQA found that RMIT has been successful in selecting credible and responsive partners. There is good communication between RMIT staff and the partner staff who handle the programs. Students and the external stakeholders are positive about the quality and value of the RMIT programs. Moderation of marking of assignments and final examinations is well in place. RMIT pays attention to achieving equivalence of outcomes to the programs taught in Melbourne. To ensure equivalence of outcomes between onshore and offshore programs, RMIT has strengthened the requirements of offshore partners and developed an offshore teaching model to be implemented in all partnerships. Strategic decisions have been made to exit from partnerships that would not be able to meet these requirements and exit strategies are in place to facilitate teach out arrangements. However, there are areas that need further attention, such 2 Australian Universities Quality Agency 2009

7 RMIT University Audit Report as ensuring clarity on the proposed offshore teaching model among the partners, integrating industry engagement and internationalisation in the TNE programs, and monitoring of TNE contracts. RMIT is addressing some of these areas and this Report affirms those actions and makes further recommendations. RMIT sets and monitors standards through the Academic Standards Framework. Under Internationalisation this Framework specifies criteria to be met in relation to entry level and entry pathways, including English language levels. It also has key measures related to graduate destinations, staff induction and support and offshore program development, approval and delivery and international research partnerships. In summary, the Panel formed the view that RMIT s attention to standards and performance monitoring in internationalisation is well on track Theme 2: Industry Engagement RMIT s strong relationship with industry has further been enhanced by its Industry Engagement Strategy. The comprehensive framework of the Strategy provides models of engagement with targeted industries and is progressing under the leadership of the Vice Chancellor and the senior executive team. There is a broad awareness of this approach, with staff at all levels strongly engaged. The Strategy supports the University well in acting on its strategic priorities. AUQA commends RMIT for its success in building strategic partnerships and for the way it has facilitated research and educational opportunities for students and staff with the industries nationally and internationally. WIL is one of the major initiatives of the Industry Engagement Strategy. While WIL has received explicit attention in the higher education programs, staff seem to have assumed that WIL is already embedded within all TAFE programs. As a result, the enthusiasm and excitement found in the higher education operations to implement WIL is lacking in the TAFE operations. RMIT should address this discrepancy to strengthen the benefits of cross sectoral engagement in the WIL strategy. RMIT is continuing to develop key performance indicators (KPIs) to track progress and successful outcomes against identified strategies and initiatives. It proposes to report against them from July 2009 and AUQA endorses this intention National Protocols The National Protocols for Higher Education Approval Processes require all universities to meet a range of criteria, in particular, nationally agreed criteria for all higher education institutions (A1 to A10) and criteria for universities (D1 to D5). For the 2009 audits onwards, AUQA required each auditee to complete a preliminary desk review of its operations against the criteria in the National Protocols for Higher Education Approval Processes. In response to that, RMIT produced an assessment of the extent of its compliance with each of the criteria in Protocols A and D and provided a list of the evidence used by the University to make the assessment. The desk review was considered by AUQA in developing the scope of the main Audit Visit and it also informed the Panel s attention to the audit scope. On the evidence considered by the Panel, AUQA believes that RMIT satisfies the criteria of the National Protocols. Australian Universities Quality Agency

8 Audit Report RMIT University Other External Reference Points RMIT has used a number of other reference points derived from the following to monitor its performance: Australian Qualifications Framework Australian Quality Training Framework Institution Assessment Framework Portfolio Course Experience Questionnaire Graduate Destination Survey Postgraduate Research Experience Questionnaire RMIT also participates in a number of benchmarking processes within particular areas through the following: Council of Australian University Librarians Tertiary Education Facilities Management Association Council of Australian University Directors of IT Australian Universities International Directors Forum Australian Technology Network In addition to the National Protocols, the University has considered the AVCC (now Universities Australia) Provision of Education to International Students: Code of Practice and Guidelines for Australian Universities while developing its policies and procedures. The University has recently reviewed its compliance with the Education Services for Overseas Students Act 2000 (ESOS Act), and the National Code of Practice for Registration Authorities and Providers of Education and Training to Overseas Students. With reference to these external reference points and benchmarking initiatives, AUQA has not identified any issues that might affect the quality of RMIT s educational activities. 1.2 Institutional Context Institutional Profile Founded as the Working Men s College in 1887, RMIT University is known for its integrated approach to VET and higher education in education and training. From an intake of 600 students in 1887, RMIT has grown to a large university of more than 65,000 students. It also has a high proportion of international students (in the Australian Technology Network (ATN) cohort), approximately 23,000, drawn from more than 100 countries. RMIT has three Melbourne campuses City, Brunswick and Bundoora, a small regional campus as well as two campuses in Vietnam Ho Chi Minh City and Hanoi. As a dual sector tertiary institution, in 2008, RMIT offered 842 higher education and vocational programs, ranging from apprenticeship training to doctoral programs. 4 Australian Universities Quality Agency 2009

9 RMIT University Audit Report Student Enrolment (Headcounts) by Campus, 2008 Campus Higher Education VET Total City 25,814 16,275 42,089 Bundoora ,057 Brunswick 261 2,357 2,618 Regional Victoria Off Campus Australia 1,109 1,077 Offshore (TNE) 10,600 1,049 11,649 Vietnam 3,312 3,312 RMIT Total 46,951 21,162 68,113 Source: Portfolio p12 (Preliminary data as at 24 November; VET based on confirmations only as at reference date). Staff Full time Equivalent by Function, 2008 Staff Category Female Male Total Academic Teaching and Research Research only VET/TAFE Executive General College and School support Central support TOTAL Source: Portfolio p13 (exclude casuals, data as at 30 September 2008: Source: People and Culture) Strategic Context of the Audit This is the first audit of a dual sector tertiary institution that covered both higher education and vocational education and training (VET) and was conducted jointly by two quality assurance bodies. The Victorian Registration and Qualifications Authority (VRQA) established under the Victorian Education and Training Reform Act 2006 is responsible for the regulation of education and training providers, from home schooling to higher education, and course accreditation processes in Victoria. The Australian Universities Quality Agency (AUQA) is responsible for the quality audits of universities, government accreditation authorities, and other higher education institutions in Australia. As a result of their long standing cooperation in the theory and practice of quality assurance in technical and higher education, the VRQA and AUQA entered into a Memorandum of Cooperation on 1 July 2008 to establish cooperative and collaborative mechanisms, for the mutual benefit of both parties and their stakeholders. Within the broad framework of the Memorandum of Cooperation, where feasible, the two agencies have agreed to collaborate on audits of institutions that include both technical and higher education activities, both self accrediting and non self accrediting institutions. Under this provision, the two agencies worked with RMIT to identify the scope and the process for the joint audit. This was made possible due to RMIT s willingness to undertake this challenging task and the confidence it had in its well integrated functioning of higher education and VET operations. The joint audit process involved VRQA nominating an auditor to the joint panel who was also supported by a researcher to review sampled VET qualifications. The scope of the joint audit was Australian Universities Quality Agency

10 Audit Report RMIT University facilitated to a large extent by the theme of Industry Engagement. The new strategic focus of RMIT in partnering with industries through the implementation of the Industry Engagement Strategy gave adequate coverage to the AQTF standards pertaining to development of training and assessment strategies in consultation with the industry. Acknowledging the integrated functioning of higher education and VET operations at RMIT, this Report relates to whole of University processes; it differentiates between higher education and VET only where necessary. The audit conclusions given in the Report need to be read in this context. 1.3 Commendations, Affirmations and Recommendations This Report contains commendations, affirmations and recommendations. A commendation refers to the achievement of a stated goal, or to some plan or activity that has led to, or appears likely to lead to, the achievement of a stated goal, and which in AUQA s view is particularly significant. A recommendation refers to an area in need of attention, whether in respect of approach, deployment or results, which in AUQA s view is particularly significant. Where such matters have already been identified by RMIT, with evidence, they are termed affirmations. High priority recommendations and affirmations are marked urgent. It is acknowledged that recommendations in this Audit Report may have resource implications. The themes for Cycle 2 audits are chosen for their risk potential and at least one theme reflects the institution s own assessment of its developmental and strategic needs. For this reason, Cycle 2 audit reports may contain more recommendations and affirmations than commendations. This is particularly important in the case of RMIT since the theme Industry Engagement, although based on a long tradition of working with industry, has a new strategic focus to engage with targeted industries. It was initiated only in 2006 and implementation is still evolving. The other theme, Internationalisation, is also marked by many developments in the offshore campus in Vietnam and the strategic review of the TNE partnerships. As a result, AUQA has made many affirmations and recommendations; as was the case with AUQA s Cycle 1 audits, this Report aims to assist the University to enhance the quality and standards of its operations. Commendations 1. AUQA commends RMIT for the effective changes it has introduced since 2003 in its planning, resource allocation, and review systems AUQA commends the Vice Chancellor and the Executive Team of RMIT for their strong leadership and high level of communication which has resulted in a clearer understanding of the responsibilities of staff at all levels, and a demonstrable improvement in staff morale AUQA commends RMIT for its establishment of high level research institutes to focus resources with the aim of achieving strong international recognition in areas of strength AUQA commends RMIT for its comprehensive support services and the successful mentoring programs which provide effective peer support to international students AUQA commends the University for its characterisation of RMIT Vietnam as a hub in Asia to facilitate two way mobility between Vietnam and Melbourne and between the other Asian partners Australian Universities Quality Agency 2009

11 RMIT University Audit Report 6. AUQA commends RMIT for developing a comprehensive framework for enhancing its already strong relationship with industry and identifying the concept of strategic partnerships AUQA commends RMIT for its approach to Work Integrated Learning that provides valuable educational and research opportunities to students and staff through initiatives such as the RMIT International Industry Experience and Research Program...33 Affirmations 1. AUQA affirms the intentions of RMIT towards facilitating the processes for commercialisation of research and strengthening support for research staff (urgent) AUQA affirms RMIT s decision to establish a School of Graduate Research to improve support for graduate students and urges the University to expedite the process AUQA affirms RMIT s intention to develop an action plan for internationalisation and advises attention to coordination of interrelated strategies for onshore and offshore international activities AUQA affirms RMIT s support for the international experience of its students and staff but urges the establishment of a data collection system to measure and track all forms of student and staff mobility AUQA affirms the actions being taken by RMIT to implement rigorous procedures in the strategic selection of partners and in managing contracts for transnational education partnerships in higher education and TAFE programs AUQA affirms RMIT s plan to introduce a formal mechanism to capture industry feedback and to strengthen attention to indicators that better measure industry engagement Recommendations 1. AUQA recommends that RMIT consider rationalising the various research committees to consolidate the research effort and to ensure consistent understanding of research expectations among staff AUQA recommends that RMIT consider further ways to promote staff attention to continuous improvement of the quality of teaching and learning, including greater use of the available processes and indicators AUQA recommends that RMIT ensure better alignment between planning and tracking to support the sound planning framework it has in place with timely monitoring of the implementation of plans at the process, operational and strategic levels AUQA recommends that RMIT strengthen further its strategies for sharing of good practices in internationalisation of the curriculum, to ensure implementation across the University in both higher education and VET operations (urgent) AUQA recommends that RMIT ensure timely declaration of examination results in Vietnam...24 Australian Universities Quality Agency

12 Audit Report RMIT University 6. AUQA recommends that implementation of the Work Integrated Learning Policy is given explicit attention across the full range of VET programs of RMIT to ensure that experiential learning is embedded in each VET program Australian Universities Quality Agency 2009

13 RMIT University Audit Report 2 MATTERS FROM AUDIT CYCLE 1 Recognising the importance of quality enhancement and improvement, the audit considers whether the recommendations and affirmations in the 2003 Cycle 1 AUQA audit report have been addressed. A sample of recommendations and affirmations is selected and checked. As well, AUQA seeks evidence of the increasing effectiveness of the institution s quality assurance and improvement system. In its Performance Portfolio, RMIT provided a report on progress in implementing recommendations from the 2003 Cycle 1 audit. The Panel s overall observations in this regard are given in section 2.1. In addition to the overall assessment of the improvement system in place, the Panel identified the recommendations on research and research training for further follow up. Actions taken on seven recommendations on research and research training were probed in detail during the course of the audit and the relevant comments are given in section 2.2. Several other recommendations from the 2003 AUQA audit related to the two themes of this audit are also discussed in sections 3 and 4 as appropriate. Comments on quality assurance systems and processes are included in section Implementation of Cycle 1 Recommendations Major changes have occurred within the University since the 2003 AUQA audit although most of this has occurred only during the last three years. There has been a substantial change in the membership of the Executive Team; most members, including the Vice Chancellor (VC), were appointed after the 2003 AUQA audit. Since the appointment of a new VC in 2005, RMIT s attention to the 2003 AUQA audit recommendations has accelerated in a systematic manner. At present, there is exponential increase in the pace at which improvement plans are being implemented. The 2003 AUQA audit recommendations went to the core of how the University was organised and there was little untouched by the recommendations. Addressing those concerns required changes in all aspects of the University operations. RMIT considered a review of the management structure and lines of responsibility as a prerequisite for dealing with those major changes. Moreover, a concern identified during the 2003 AUQA audit that affected a number of issues, was the lack of clarity of relative roles and responsibilities of various boards and management committees. Subsequently, the University has remodelled its governance and management processes and systems, clarified the management responsibilities, the planning process, reporting relationships and roles of various committees. Senior management and committee restructuring comes through as appropriate and effective, although the Panel still has concerns in the research and research training area. There has been a high level response to the recommendations on Leadership and Strategic Planning (recommendations 1 6). Planning is comprehensive. There is a structured planning framework that uses the RMIT 2010 Strategic Plan as its central reference and contains a number of new directions and areas of focus, as well as a revised set of KPIs. The top down/bottom up strategic plans, linked to the academic plan, linked to business plans, linked to resource allocation and risk management, are well described and articulated, and they appear to be effective. Australian Universities Quality Agency

14 Audit Report RMIT University The Panel followed a number of trails through the planning hierarchy to examine effectiveness of implementation and alignment from business and academic plans to school, group and individual work plans as depicted in the framework. Whilst the overall design of these plans is exemplary, some aspects of the implementation process related to tracking improvement need attention. The University is candid about identification of areas requiring further improvements. Many of the improvement strategies identified are due for completion in 2009 with a seemingly heavy workload scheduled for the year. For more comments on how well the University is tracking the improvements see section 2.3. The changes since 2003 in the resource allocation and review systems are significant. The resource allocation system appears to be coherent and it incorporates international good practice on budget parameter setting (driven by profiles), expenditure allocation (based on the Strategic Plan as well as continuing unit needs), the ability to meet forecasts (within 1%), and (recently) incentives for operating units to enhance their margins through efficient operations. The cyclic organisational review, business plan review, and program annual review processes are fit for purpose and appear to be operating effectively. RMIT follows the practice of having external council members chairing the organisational reviews. Although beneficial in terms of Council visibility and knowledge of the University, given the governing role of the Council, this practice could lead to situations of potential conflict and RMIT has to be alert to this possibility. Overall, changes made to planning, resource allocation and review systems have worked well. Commendation 1 AUQA commends RMIT for the effective changes it has introduced since 2003 in its planning, resource allocation, and review systems. A major concern of the 2003 AUQA audit was that of general communication with staff. Under the strong leadership provided by the Vice Chancellor, there is now a high level of communication with staff throughout the University which has ensured a clearer understanding of the change processes and the responsibilities of staff. Staff at all levels are, in general, positive about the changes in the institution and there is significant improvement in staff morale. Participation of staff in the challenging change processes, and their support in meeting demanding targets, especially during times of financial difficulties, was acknowledged by senior management. Commendation 2 AUQA commends the Vice Chancellor and the Executive Team of RMIT for their strong leadership and high level of communication which has resulted in a clearer understanding of the responsibilities of staff at all levels, and a demonstrable improvement in staff morale. 2.2 Follow up: Recommendations on Research and Research Training A number of major recommendations of the 2003 AUQA audit are already covered in the two themes of this audit and the attention to quality assurance systems (section 2.3). The risk assessment undertaken by AUQA in the light of the progress report on Cycle 1 recommendations, RMIT s Strategic Plan, and the Institution Assessment Framework Portfolio 2007 indicated that a cluster of recommendations on research and research training needed attention. Subsequently, after considering the Performance Portfolio and the supporting documents, the Panel selected recommendations on research and research training for specific follow up during the main audit. 10 Australian Universities Quality Agency 2009

15 RMIT University Audit Report Recommendation 11 of the 2003 AUQA audit stated that RMIT University clarify to staff its overall strategy for research and research training and the key priorities therein, including ongoing work to rationalise performance indicators. It also urged the University to clarify accountabilities and responsibilities with respect to research and research training performance. RMIT has made significant progress towards clarifying the overall strategy for research and research training. The updated Research and Innovation Strategy incorporates KPIs aligned to the Strategic and Business Plan. Progress has been made on the KPIs at the business plan level but the performance database is still relatively new. Considerable success in gaining Australian Research Council Discovery Projects and Australian Research Council Linkage Projects, increase in percentage of doctorates and masters completions and publication of journal articles are positive outcomes of RMIT s attention to its research strategy (data items 5.1 and 5.2). Communication related to the research strategy has improved but is still a work in progress. Accountabilities and responsibilities with respect to research are being clarified, but concerns still exist in research training (see Recommendation 1 and Affirmation 1). There is apprehension among staff with respect to definition of outcomes in fields, where publication in Tier 1 journals is not the standard eg, Art and Design and VET areas. Even where applicable, there are concerns that the emphasis on Tier 1 journals may produce perverse incentives for many staff in that such publication may be beyond the capability of many outside the research institutes. There were also concerns among staff that interdisciplinary research did not really qualify for points in this scheme of research assessment. It is expected that the introduction of Excellence of Research in Australia will inform research performance measures. Recommendation 12 of the 2003 AUQA audit stated that, in considering the most appropriate research committee management structure, RMIT University ensure that it is able to provide sufficient opportunities for appropriate stakeholder input with minimum complexity. To address this Recommendation, the committees that support research have been restructured. However, the research committee does not appear to add significant value beyond supporting research supervision. Furthermore, the articulation between the college and University research committees was not clear. The University states that including the Pro Vice Chancellor (Research and Innovation) in the membership of the boards of each of the three academic colleges reinforces the link between the portfolio of Research and Innovation and the various committees. In spite of this linkage, there still is a great deal of variation among colleges in research expectations of staff and support made available to them. Recommendation 1 AUQA recommends that RMIT consider rationalising the various research committees to consolidate the research effort and to ensure consistent understanding of research expectations among staff. Recommendation 13 of the 2003 AUQA audit stated that RMIT University clarify the nature of the Virtual Research and Innovation Institutes and consider re naming the scheme to better reflect its intended role. In response to this Recommendation, RMIT did a review of the virtual research and innovation institutes and carried out comprehensive planning in 2006 and 2007 to establish four new research institutes in place of the virtual research and innovation institutes. The four research institutes namely RMIT Global Cities Research Institute, RMIT Platform Technologies Research Institute, RMIT Design Research Institute, and RMIT Health Innovations Research Institute are charged with developing RMIT s capabilities at nationally and internationally leading levels. Staff and students associated with the institutes are of very high Australian Universities Quality Agency

16 Audit Report RMIT University quality and are well supported. Establishment of these research institutes to strengthen international recognition in areas of comparative advantage is a good concept and is working successfully. Commendation 3 AUQA commends RMIT for its establishment of high level research institutes to focus resources with the aim of achieving strong international recognition in areas of strength. Recommendation 14 of the 2003 AUQA audit stated that RMIT University ensure that its processes for commercialisation of research and the commissioning of major items of research equipment are such that it is able to respond to research opportunities in a timely manner. During , the University has recorded an increase in the total commercial research income (PF p129). However, support for commercialisation of research needs strengthening. An effective technology licensing program is yet to be developed, and indeed the Panel formed the view that RMIT does not consider this as a high priority for investment. There have been some good initiatives to support staff such as help with applying for grants, central invoicing and increase in the allocation to colleges of discretionary capital budget. The Panel also noted that the University offers a graduate certificate course in research commercialisation in conjunction with other ATN universities. The relevant processes within the centres and institutes appear robust, but there is too much variability among the schools. A new manager has just been appointed to improve University wide coordination of research partnerships. More has to be done to strengthen research support and develop adequate KPIs. Affirmation 1 AUQA affirms the intentions of RMIT towards facilitating the processes for commercialisation of research and strengthening support for research staff. Recommendation 15 of the 2003 AUQA audit stated that, as part of the development of the research strategy for RMIT International University Vietnam, the University explicitly consider how Melbourne staff can support and facilitate research capacity building and set agreed targets and research performance indicators so as to monitor progress. RMIT acknowledges that much of the work that has been carried out in response to this Recommendation has been focused on getting research into Vietnam in the first instance. The series of research workshops during 2002, 2004, and 2006 leading to project partnerships in Vietnam, and the collaborative scholarship funding package involving the Ministry of Education and Training and other partner institutions in Vietnam are notable beginnings. The University is aware of the challenges involved in the transition of RMIT Vietnam from a teaching only campus to one that includes research. These include sourcing adequate funding for substantial research projects, securing capable and experienced researchers who are prepared to move to Vietnam, and developing research infrastructure for research in areas other than the social sciences. A draft position paper on research development at RMIT Vietnam provides various options in relation to research at RMIT Vietnam, including whether it should pursue research or not. The Panel noted that senior management wishes to strengthen RMIT Vietnam s research capacity by employing research active staff in the future. At present there is a flat structure of staff appointed at lecturer level with an increased pay level for those who are program or course coordinators. Many initiatives mentioned in the Portfolio to enhance the research capacity at 12 Australian Universities Quality Agency 2009

17 RMIT University Audit Report RMIT Vietnam are either in the initial stages or still under negotiation. The University should quickly firm up its research strategy for Vietnam and accordingly adjust the extent and pace of capacity development activities for the Vietnam campus. Recommendation 16 of the 2003 AUQA audit stated that RMIT University establish robust supervisory performance criteria to enhance the value of the register of supervisors. Recommendation 17 of the 2003 AUQA audit is also related to ensuring that the policy on the minimum resources to be provided to higher degree by research students is consistently implemented across all academic units. The Panel found that action on research support, in particular on student supervision, has not produced consistent results across the University. It is noted that there was improvement in student satisfaction measures between 2003 and 2006; Postgraduate Research Experience Questionnaire scores improved significantly with the percent agreement on supervisory performance scale rising from 64% to 80% over that time which was above the ATN and national averages. However, the problem of variability among schools and individual supervisors remains. The research committees have been restructured to address a part of this problem but a lot more needs to be done to address the Recommendation sufficiently. The Panel was advised of many actions proposed to be implemented in the near future to ensure enhanced and consistent support to research students. It has plans to establish a School of Graduate Research to consolidate administration and support for research students and a beginning has already been made by appointing a Dean of Graduate Research. The University should move quickly on implementation of the other proposed actions. The University may like to consider whether the implementation of the strategies will be hampered by the fact that the new Dean of Graduate Research is a part time appointment. Affirmation 2 (urgent) AUQA affirms RMIT s decision to establish a School of Graduate Research to improve support for graduate students and urges the University to expedite the process. 2.3 Quality Systems and Processes Quality Assurance Internal and external reviews of compliance with the AQTF Standards and the ESOS Act have resulted in implementation of enhanced data management and better alignment of institutionwide systems and policies. RMIT is undertaking genuine internal reviews and acting on external review reports to identify and plan improvements. The RMIT document Summary of Key Activities with AQTF Essential Standards indicates that regular reviews of staff qualifications at school and college level have been conducted and improvements made where necessary (pp13, 17). However, the audit found that work plans, the primary vehicle for recording and monitoring the professional development of staff so as to maintain vocational currency as well as training and assessment competencies, were not uniformly well managed across the four programs audited. There was evidence that staff were participating in professional development, upgrading and maintaining their vocational competence and/or undertaking the Graduate Certificate in Tertiary Teaching and Learning (some with support from RMIT) to enable them to teach across both sectors. However the sample audit suggests that improvement to human resource management processes is needed Australian Universities Quality Agency

18 Audit Report RMIT University to ensure that professional development is aligned systemically to the maintenance of current vocational and training and assessment competencies. The Portfolio states that Cyclical organisational reviews introduced in 2004 (replacing Faculty Review) are now central to RMIT s quality assurance and improvement processes and contribute to RMIT s alignment with AQTF Essential Standard 3.1 (PF p36). The Quality Consultancy Unit manages the cyclical organisational review process and provides policy support to the Deputy Vice Chancellor (Academic) (DVC (A)) and others. However, the Panel found that the findings from these review activities could be used more effectively in enhancing teaching and learning across the University. The Panel noted the range of strategies in place to improve teaching and learning, including $500 million investment in teaching facilities over the next three years. Student feedback measures of good teaching and overall satisfaction have improved since 2004 although they are still below the sector and cohort measures. RMIT will need to continue to focus efforts to effect further improvement in teaching and learning in order to meet the targets set in the University s Strategic Plan. The Panel also found that staff see improvement primarily in terms of student satisfaction, and pay less attention to the other relevant available KPIs for the improvement of teaching and learning. Strengthening of staff engagement with a broader range of processes and measures would enhance their effectiveness. RMIT should identify ways to promote staff attention to quality improvement beyond student satisfaction and the enhanced attention should drive the various quality assurance and review processes across the University. Recommendation 2 AUQA recommends that RMIT consider further ways to promote staff attention to continuous improvement of the quality of teaching and learning, including greater use of the available processes and indicators Ensuring Better Alignment Between Planning and Measurement RMIT has well thought out statements of strategies and lists of actions to be taken at multiple levels of the organisation. However, the KPIs need to be better aligned with the strategies and actions. Since the Cycle 1 audit, a new set of University KPIs has been developed. The University has listed 35 KPIs with targets for each. The summary of Performance against KPIs presented at a Vice Chancellor s Executive (VCE) Retreat in September 2008, lacks adequate explanations, analysis or identified actions. The strategic KPIs do not provide sufficient information to show how the University is performing. While some of the indicators are good, more thought should be given to what senior management and Council really need to know to track University performance in relation to the Strategic Plan. The Panel formed the view that RMIT has articulated better what needs to be done than at tracking what has, or has not been done at least in areas not of such high priority and large size as to warrant a formal administrative structure. With senior management being busy with high priority actions, there are change strategies that are pushed down to levels where people are not appropriately positioned to deal with them. There are gaps in tracking outcomes in those areas, which means that sometimes signals about these gaps do not rise up through the organisation for timely monitoring. 14 Australian Universities Quality Agency 2009

19 RMIT University Audit Report While senior management is very open about potential shortcomings and the need to rectify them, and eager for advice about how to do so, there is less openness in certain areas below the senior team. There seemed to be an assumption among staff that because something has been delegated by senior management that it is being dealt with effectively. The Panel believes that this is not always carried through. At the operational level, staff are generally enthusiastic, capable, and eager to solve problems. Overall, good progress has been made on the planning framework and development of KPIs but, to effect further improvements, RMIT needs to ensure better alignment between planning and tracking. Recommendation 3 AUQA recommends that RMIT ensure better alignment between planning and tracking to support the sound planning framework it has in place with timely monitoring of the implementation of plans at the process, operational and strategic levels. 2.4 Approach to Academic Standards and Benchmarking RMIT developed an Academic Standards Framework in 2007 in consultation with other members of the ATN. The ATN Academic Standards Framework has an exclusive focus on learning and teaching quality and it consists of: a whole of university conceptual framework a suite of indicators derived from the framework a shared approach to gathering data to inform these measures (PF p57). RMIT developed its Academic Standards Framework based on this ATN model but extended it to include standards from RMIT policy and planning documents for all core academic activities. An external consultant s report (2008) on institutional level qualitative indicators related to academic governance, academic management and policies and academic culture commented positively on the progress made by RMIT. In relation to accessibility, clarity and comprehensiveness of the academic policies, RMIT performs on at least an equal footing with its selected comparators in most respects. The Panel was advised that the Framework is a living document and that work on these recommendations to improve the Framework, and its implementation, is being monitored by the DVC (A). The RMIT Institution Assessment Framework Portfolio 2008 contains substantial quantitative data on a range of indicators. RMIT, as is the case with many other universities, has expressed concerns about the lagged data generated by the Course Experience Questionnaire (CEQ). This is a sector wide issue and to overcome this difficulty, RMIT administers an internal survey, namely the Course Experience Survey that uses the same CEQ questions, but is administered immediately following completion of each course. In spite of the extensive work done towards administering thousands of surveys each year and analysing survey data, the Panel saw examples of analysis of the data that contain many subjective comments without sufficient interpretation and actions to follow. RMIT benchmarks its performance, formally and informally, in a number of ways. RMIT has compared its performance internationally against high performing institutions on identified aspects. It also participates in benchmarking processes within particular areas through the following: Australian Universities Quality Agency

20 Audit Report RMIT University Council of Australian University Librarians Tertiary Education Facilities Management Association Council of Australian University Directors of IT Australian Universities International Directors Forum Australian Technology Network RMIT is actively engaged in many benchmark initiatives with the ATN members and the development of the Academic Standards Framework is one such example. Analysing the CEQ data, graduate and alumni outcomes and KPIs in specific areas are a few other initiatives to mention. Overall, RMIT s approach to standards and benchmarking is well on track. There is good progress on the planning framework and to effect further improvements, RMIT needs to strengthen alignment between planning and tracking. 16 Australian Universities Quality Agency 2009