eportfolios in Australian higher education: Graduate attributes, standards and employability Assuring Graduate Capabilities Professor Beverley Oliver National Teaching Fellow http://tiny.cc/boliver
US, UK, Canada, Europe ME Change leadership YOU Change leadership working within our contexts to convince ourselves and others to do things differently
Tertiary Education 39 self-accrediting universities research undergraduate = largely professionally oriented highly casualized ageing academic staff ~45K students Higher Education ~1.1m students Vocational Education ~1.7m students
Level of study by 1.1m students postgrad undergrad International
What 0.8m undergrads study International
government funding drivers high accountability: bang for buck 39 self-accrediting universities (2 private) (many courses externally accredited) international student downturn widening participation protect brand Australia increasing competition from ~140 private providers
Mission-based compacts Performance funding Quantitative indicators 1. Participation/social inclusion low socio-economic 2. Student experience University Experience Survey 3. Quality of learning outcomes CLA teaching quality
Tertiary Education Standards and Quality Agency TEQSA The My University website: course informa3on campus facili3es student/staff ra3os student sa3sfac3on surveys graduate skills and outcomes fees, student services quality of teaching and learning outcomes
Australian Qualifications Framework
in other words... generic capabilities graduate attributes competencies essential learning outcomes graduate outcomes [but which ones? and how high?]
! Australian universities' graduate attribute statements (May 2011)
What about graduate employability? What are the common clusters of graduate attributes? COMMUNICATION Written and oral communication THINKING Critical and analytical and creative and reflective thinking PROBLEM SOLVING Problem-solving, generating ideas and innovative solutions INFORMATION LITERACY Use technology to find, evaluate and create information SELF-MANAGEMENT Learning and working independently TEAMWORK Learning and working collaboratively CIVIC ENGAGEMENT Ethical and inclusive engagement with communities, cultures and nations
What is graduate employability? a set of achievements skills, understandings, personal attributes aka generic skills, graduate qualities, attributes, capabilities competencies... that makes graduates more likely to gain employment and be to the benefit of successful in their chosen occupations themselves, the workforce, the community and economy Mantz Yorke
Evidence as presented in Resume Academic transcript Mary James Bachelor of XYZ Introducing Mary James Psychology 100: High distinction Science 120: Credit French 134: Pass Mathematics 156: Distinction Blah blah Blah blah Blah blah Blah blah Blah blah Blah blah Blah blah This is to certify that Mary James has just scraped through the Doctor of Philosophy
How can we assure outcomes and standards?
Assurance of learning for graduate employability Plan enhancements Benchmarking portfolios: 6. Benchmark, plan enhancements --and start again Determine capabilities 1. Determine the capabilities and standards for early professional success Evaluate outcomes Course review portfolios: Analyse indicators of course quality including perceptions of graduate achievement of capabilities 5. A 360-degree evidence-based approach for curriculum enhancement 4. 3. 2. Map inputs Map intended development and assessment of capabilities throughout the curriculum Student portfolios: Evaluate achievement of capabilities in portfolio approaches to teacher, self and peer assessment Support student achievement of the capabilities through work-integrated learning
1. a set of achievements skills, understandings, personal attributes capabilities that makes graduates more likely to gain employment...
Professional accreditation competencies Program learning outcomes COMMUNICATION THINKING But how well? PROBLEM SOLVING Learning standards INFORMATION LITERACY SELF-MANAGEMENT TEAMWORK and provide examplars This is not a scoring rubric. CIVIC ENGAGEMENT Degree program teams make degree program rubrics to specify standards in generic capabilities
Relevant AAC&U Rubrics COMMUNICATION Written Communication/ Oral Communication Written and oral communication THINKING Critical Thinking/Inquiry and Analysis Critical and analytical and creative and reflective thinking PROBLEM SOLVING Problem Solving/ Creative Thinking/ Quantitative Literacy Problem-solving, generating ideas and innovative solutions INFORMATION LITERACY Information Literacy/ Reading Use technology to find, evaluate and create information SELF-MANAGEMENT Lifelong Learning/Integrative Learning Learning and working independently TEAMWORK Teamwork Learning and working collaboratively CIVIC ENGAGEMENT Civic Engagement/Ethical Reasoning/Intercultural Ethical and inclusive engagement with communities, cultures and nations
2. Map capability development and assessment 3. Map work integrated learning
In summary: six visual analyses 1. Graduate Attributes You are welcome to have this tool. 2. Assessment 3. Learning experiences 4. Learning resources 5. Curriculum themes 6. Career development learning http://tiny.cc/boliver
then there s the tricky bit
5. Measures Plan enhancements [TEQSA SPEAK] To what extent are capabilities demonstrated by graduates? Gather stakeholder perceptions of new graduates achievement of those capabilities Employment: got a job? GDS Self-reported: got generic skills? CEQ Map student development and assessment of capabilities throughout the curriculum S
Measures COMMUNICATION THINKING PROBLEM SOLVING INFORMATION LITERACY Collegiate Learning Assessment wri?en communica3on cri3cal thinking analy3cal reasoning problem solving CEQ Generic Skills Scale (self-reported) written communication analytic skills problem-solving skills confident unfamiliar problems SELF-MANAGEMENT plan my own work TEAMWORK work as a team member CIVIC ENGAGEMENT
Graduate Employability Indicators G GRADUATES to what extent? how important? 14 capabilities (NSSE) EMPLOYERS TEACHING TEAMS
Graduate, employer and course team (% agreement 'very important') to early professional success
You are welcome to use this tool. http://tiny.cc/boliver
measure ~ summative/quantitative judge ~ formative/qualitative Evidence Mantz Yorke
bottom up marks grades credits 50% measurement 50% 50% 50%
top down learning outcomes judgement
The key question to the student: How have you satisfied, through your work, the aims [and standards] stated for your particular program of study? Mantz Yorke
the student makes a case that they merit the award evidence: a mixture of marks or grades...qualitative assessments of performance in naturalistic settings (such as work placements), and claims of achievements that are not formally assessable but nevertheless supported by evidence. The making of claims of this sort... would require the collation of a portfolio of achievements Mantz Yorke
4. Get students focused on collecting evidence of achievement of those standards
about me introduce yourself, your biography and goals then invite others to view, provide feedback or collaborate
my ratings rate your achievement of the graduate attributes then invite others to view, provide feedback or collaborate
my course reflect on assessments in formal units of study then invite others to view, provide feedback or collaborate
my employment build your resume, reflect on professional development then invite others to view, provide feedback or collaborate
my journals keep journals on any topic associated with your learning then invite others to view, provide feedback or collaborate
examples
my showcases then publish polished showcase achievements at itunes U invite others to view, provide feedback or collaborate
Assurance of learning for graduate employability Plan enhancements Benchmarking portfolios: Plan and monitor enhancements --start again 6. Determine capabilities 1. Determine capabilities and standards for early professional success Evaluate outcomes See what new graduates, employers and 5. A 360-degree evidence-based approach for curriculum enhancement 4. teaching staff say are the most Create, manage, important share self- and peer assessment of the capabilities with externals, teaching staff and peers 3. 2. Map inputs course Collect evidence of your learning in work integrated learning contexts See where the capabilities are developed and assessed in your
Clarify the minimum standards in generic capabilities Use portfolios of evidence to judge achievement of the standards
assess a whole eportfolio? an analogy...
Capstone Portfolio Assessment Yr 1 Yr 2 Yr 3 Capability X: synthesis of evidence Capability 2: synthesis of evidence For marks or credit or not Formative or summative or both Teacher or self or peer assessed or all How might we make this...valid, Reliable, Robust, Efficient and Affordable? Auditable evidence of capability standards
So: wrapping up WIIFM
marks grades credits learning outcomes transcripts portable HEAR, AHEGS digital warranted/not warranted evidence that helps me achieve my life goals
so what s an eportfolio? does it matter?
assuring learning through authentic experiential evidence-based approaches
http://tiny.cc/boliver