Assuring Graduate Capabilities

Transcription

1 eportfolios in Australian higher education: Graduate attributes, standards and employability Assuring Graduate Capabilities Professor Beverley Oliver National Teaching Fellow

2 US, UK, Canada, Europe ME Change leadership YOU Change leadership working within our contexts to convince ourselves and others to do things differently

3 Tertiary Education 39 self-accrediting universities research undergraduate = largely professionally oriented highly casualized ageing academic staff ~45K students Higher Education ~1.1m students Vocational Education ~1.7m students

4 Level of study by 1.1m students postgrad undergrad International

5 What 0.8m undergrads study International

6 government funding drivers high accountability: bang for buck 39 self-accrediting universities (2 private) (many courses externally accredited) international student downturn widening participation protect brand Australia increasing competition from ~140 private providers

7 Mission-based compacts Performance funding Quantitative indicators 1. Participation/social inclusion low socio-economic 2. Student experience University Experience Survey 3. Quality of learning outcomes CLA teaching quality

8 Tertiary Education Standards and Quality Agency TEQSA The My University website: course informa3on campus facili3es student/staff ra3os student sa3sfac3on surveys graduate skills and outcomes fees, student services quality of teaching and learning outcomes

9 Australian Qualifications Framework

10 in other words... generic capabilities graduate attributes competencies essential learning outcomes graduate outcomes [but which ones? and how high?]

11 ! Australian universities' graduate attribute statements (May 2011)

12 What about graduate employability? What are the common clusters of graduate attributes? COMMUNICATION Written and oral communication THINKING Critical and analytical and creative and reflective thinking PROBLEM SOLVING Problem-solving, generating ideas and innovative solutions INFORMATION LITERACY Use technology to find, evaluate and create information SELF-MANAGEMENT Learning and working independently TEAMWORK Learning and working collaboratively CIVIC ENGAGEMENT Ethical and inclusive engagement with communities, cultures and nations

13 What is graduate employability? a set of achievements skills, understandings, personal attributes aka generic skills, graduate qualities, attributes, capabilities competencies... that makes graduates more likely to gain employment and be to the benefit of successful in their chosen occupations themselves, the workforce, the community and economy Mantz Yorke

14 Evidence as presented in Resume Academic transcript Mary James Bachelor of XYZ Introducing Mary James Psychology 100: High distinction Science 120: Credit French 134: Pass Mathematics 156: Distinction Blah blah Blah blah Blah blah Blah blah Blah blah Blah blah Blah blah This is to certify that Mary James has just scraped through the Doctor of Philosophy

15 How can we assure outcomes and standards?

16 Assurance of learning for graduate employability Plan enhancements Benchmarking portfolios: 6. Benchmark, plan enhancements --and start again Determine capabilities 1. Determine the capabilities and standards for early professional success Evaluate outcomes Course review portfolios: Analyse indicators of course quality including perceptions of graduate achievement of capabilities 5. A 360-degree evidence-based approach for curriculum enhancement Map inputs Map intended development and assessment of capabilities throughout the curriculum Student portfolios: Evaluate achievement of capabilities in portfolio approaches to teacher, self and peer assessment Support student achievement of the capabilities through work-integrated learning

17 1. a set of achievements skills, understandings, personal attributes capabilities that makes graduates more likely to gain employment...

18 Professional accreditation competencies Program learning outcomes COMMUNICATION THINKING But how well? PROBLEM SOLVING Learning standards INFORMATION LITERACY SELF-MANAGEMENT TEAMWORK and provide examplars This is not a scoring rubric. CIVIC ENGAGEMENT Degree program teams make degree program rubrics to specify standards in generic capabilities

19 Relevant AAC&U Rubrics COMMUNICATION Written Communication/ Oral Communication Written and oral communication THINKING Critical Thinking/Inquiry and Analysis Critical and analytical and creative and reflective thinking PROBLEM SOLVING Problem Solving/ Creative Thinking/ Quantitative Literacy Problem-solving, generating ideas and innovative solutions INFORMATION LITERACY Information Literacy/ Reading Use technology to find, evaluate and create information SELF-MANAGEMENT Lifelong Learning/Integrative Learning Learning and working independently TEAMWORK Teamwork Learning and working collaboratively CIVIC ENGAGEMENT Civic Engagement/Ethical Reasoning/Intercultural Ethical and inclusive engagement with communities, cultures and nations

20 2. Map capability development and assessment 3. Map work integrated learning

21 In summary: six visual analyses 1. Graduate Attributes You are welcome to have this tool. 2. Assessment 3. Learning experiences 4. Learning resources 5. Curriculum themes 6. Career development learning

22 then there s the tricky bit

23 5. Measures Plan enhancements [TEQSA SPEAK] To what extent are capabilities demonstrated by graduates? Gather stakeholder perceptions of new graduates achievement of those capabilities Employment: got a job? GDS Self-reported: got generic skills? CEQ Map student development and assessment of capabilities throughout the curriculum S

24 Measures COMMUNICATION THINKING PROBLEM SOLVING INFORMATION LITERACY Collegiate Learning Assessment wri?en communica3on cri3cal thinking analy3cal reasoning problem solving CEQ Generic Skills Scale (self-reported) written communication analytic skills problem-solving skills confident unfamiliar problems SELF-MANAGEMENT plan my own work TEAMWORK work as a team member CIVIC ENGAGEMENT

25 Graduate Employability Indicators G GRADUATES to what extent? how important? 14 capabilities (NSSE) EMPLOYERS TEACHING TEAMS

26 Graduate, employer and course team (% agreement 'very important') to early professional success

27 You are welcome to use this tool.

28 measure ~ summative/quantitative judge ~ formative/qualitative Evidence Mantz Yorke

29 bottom up marks grades credits 50% measurement 50% 50% 50%

30 top down learning outcomes judgement

31 The key question to the student: How have you satisfied, through your work, the aims [and standards] stated for your particular program of study? Mantz Yorke

32 the student makes a case that they merit the award evidence: a mixture of marks or grades...qualitative assessments of performance in naturalistic settings (such as work placements), and claims of achievements that are not formally assessable but nevertheless supported by evidence. The making of claims of this sort... would require the collation of a portfolio of achievements Mantz Yorke

33 4. Get students focused on collecting evidence of achievement of those standards

34 about me introduce yourself, your biography and goals then invite others to view, provide feedback or collaborate

35 my ratings rate your achievement of the graduate attributes then invite others to view, provide feedback or collaborate

36 my course reflect on assessments in formal units of study then invite others to view, provide feedback or collaborate

37 my employment build your resume, reflect on professional development then invite others to view, provide feedback or collaborate

38 my journals keep journals on any topic associated with your learning then invite others to view, provide feedback or collaborate

39 examples

40 my showcases then publish polished showcase achievements at itunes U invite others to view, provide feedback or collaborate

41 Assurance of learning for graduate employability Plan enhancements Benchmarking portfolios: Plan and monitor enhancements --start again 6. Determine capabilities 1. Determine capabilities and standards for early professional success Evaluate outcomes See what new graduates, employers and 5. A 360-degree evidence-based approach for curriculum enhancement 4. teaching staff say are the most Create, manage, important share self- and peer assessment of the capabilities with externals, teaching staff and peers Map inputs course Collect evidence of your learning in work integrated learning contexts See where the capabilities are developed and assessed in your

42 Clarify the minimum standards in generic capabilities Use portfolios of evidence to judge achievement of the standards

43 assess a whole eportfolio? an analogy...

44

45 Capstone Portfolio Assessment Yr 1 Yr 2 Yr 3 Capability X: synthesis of evidence Capability 2: synthesis of evidence For marks or credit or not Formative or summative or both Teacher or self or peer assessed or all How might we make this...valid, Reliable, Robust, Efficient and Affordable? Auditable evidence of capability standards

46 So: wrapping up WIIFM

47 marks grades credits learning outcomes transcripts portable HEAR, AHEGS digital warranted/not warranted evidence that helps me achieve my life goals

48 so what s an eportfolio? does it matter?

49 assuring learning through authentic experiential evidence-based approaches

50