Presentation of the article E-portfolio: an assessment tool for online courses Exam portfolio by Kristoffer Aas E-assessment, 2014 Table of content Presentation of the authors... 2 Abstract... 2 E-portfolios - an adaption of the original concept... 2 Characteristics of e-portfolios... 3 Application of the e-portfolio with learning objects as an assessment method... 4 Evaluation and results... 4 Connecting the article to pre known theories... 5 Reflections on how to apply the idea of this paper in a relevant educational context... 6 Personal comments on strengths and weaknesses... 7 References... 8 Attachment... I
Presentation of the authors Robin Mason is Professor of Educational Technology, Chris Pegler is a lecturer and doctoral student, and Martin Weller is a Senior Lecturer, all at the Institute of Educational Technology, The Open University. They are researchers in the area of the pedagogy of learning objects. Abstract This article considers the various uses of e-portfolios in an educational context and looks at the particular characteristics of the electronic version of portfolios. It then focuses on the application of the e-portfolio as an assessment method. A case is made for the use of the e- portfolio as an appropriate end of course assessment process where learning objects are the basis of the course design. Evaluation data from such a course is presented. It is a postgraduate online course run by the Institute of Educational Technology at the Open University. Conclusions are drawn from the evaluation about the appropriateness of e-portfolios as an end of course assessment method. E-portfolios - an adaption of the original concept Mason, Pegler and Weller highlight the specific features that electronic access and digitisation provide to the portfolio process compared to the paper-based versions of portfolios. The authors separate educational portfolios into different uses, prepared for diverse purposes and utilizing various types of assets producing a scientific categorization of electronic portfolios in order to discuss and classify examples: those for developmental purposes those for presentation purposes those for assessment purposes 2
Some of the uses of e-portfolios are for short-term purposes, but the basic concept is where learners contribute continually to their portfolio throughout their learning period and the content is assessed during this time and/or in the end. Characteristics of e-portfolios Looking at course specific applications only, a various preferences of e-portfolios can be recognized for each of the stages (collection, selection, reflection, projection, presentation) associated with portfolio use in the literature (Barrett in Mason, Pegler, & Weller, 2004). Barett defines the phases as following: Collection - teachers and students learn to save artifacts that represent the successes (and "growth opportunities") in their day-to-day teaching and learning Selection - teachers and students review and evaluate the artifacts they have saved, and identify those that demonstrate achievement of specific standards (this is where most electronic portfolios stop) Reflection - teachers and students become reflective professionals, evaluating their own growth over time and their achievement of the standards, as well as the gaps in their development Projection - teachers and students compare their reflections to the standards and performance indicators, and set learning goals for the future. This is the stage that turns portfolio development into professional development and supports lifelong learning. Presentation - teachers and students share their portfolios with their peers. This is the stage where appropriate "public" commitments can be made to encourage collaboration and commitment to professional development and lifelong learning (Helen C. Barrett, 2002). Mason, Pegler and Weller describes the benefits of these stages and summarize that in terms of assessment, the e-portfolio provides the student with authentic, reflective, interactive and 3
individual features, and all of these attributes have advantages over examinations and computer-assisted, multiple choice forms of assessment (Chang in Mason et al., 2004 p. 719) Application of the e-portfolio with learning objects as an assessment method The increase in use of learning object as a course design strategy matches the growth in the application of e-portfolio. Both involve the same fundamental technology and both rely on the same components of re-use and selectivity. In this article the writers present the data and result of using e portfolio as the final assessment of a course designed in learning objects. The course they stage is run entirely online and contains 45 students from 16 different countries, and it s part of a Master s Degree programme offered by The Open University. The pedagogical approach of the programme is constructivist and student-centred, using asynchronous collaborative discussion and online activities as the core learning mode (Mason et al., 2004). The course consisted of over 100 learning objects to choose amongst, of which 55 were designated as possible option to use in the ECA (end of assessment) e-portfolio. They were obligated to select 8 learning objects to work through. They were also required to write a reflection on their choices. Most of the learning objects consisted of three elements: 1. An overview of the topic, highlighting the core issues, problems or ideas. 2. Links to further resources, web sites or journal articles for further reading. 3. One or two activities, some individual, some collaborative that form the basic experiential pedagogy of the course (Mason et al., 2004). Evaluation and results Evidence from the content of the e-portfolios submitted indicates that 90 per cent of the students engaged very satisfactorily with the portfolio statement and the concept of learning 4
objects as supporting evidence. All of the e-portfolios achieved at least a pass grade, using double marking and an external examiner. A third of students interviewed reported difficulties with the necessity to make choices: they worried that the learning objects they worked on during the course were not the best ones to submit in their portfolio; they felt overwhelmed by the number of topics. The authors pinpoint that the use of choices, in the study of the course and in the e-portfolio submission, needs further reinforcement in future presentations through greater clarity of instructions from the course team and through involvements by the tutors for students who need more support. Further, they point out that there will always be a spread of competencies amongst any student cohort, such that some students will need more support and confidence building than others. In the writers view this does not justify designing courses for the lowest common denominator. I personally think that the number of choices is satisfactory for this group, and it is used as a teaching strategy, but for younger pupils I question the benefits. As a conclusion, the authors determine that the evidence submitted support the hypothesis that e-portfolios can be a fitting assessment model for course designed in learning objects. Connecting the article to pre known theories At least two of the learning objects chosen for the e-portfolios had to result from online collaborative activities, while the rest could be from a range of individual online activities. Some of the activities involved writing; a few required PowerPoint or spreadsheet outputs, some required exploration of unfamiliar technologies; many involved searching on the web; others required online discussion or interaction amongst students in small groups (Mason et al., 2004, p. 721). Erstad (Erstad, 2009) brings forward three examples of areas where IT has been implemented in education and showed promising result: Digital portfolios, digital tools in collaborative work and peer-assessment. According to Erstad, the assessment perspective emphasize that process should be assessed as well as product, that the conception be dynamic rather than static, and that attention must be paid to the social and cultural context of both learning and assessment (Gibbs cited in Erstad, 2008, p. 182). This underpins the findings by Mason, Pegler and Weller, which points out that e-portfolios can be a suitable assessment tool. 5
Further, it supports the importance of the use of collaborative tasks in a learning process, which is integrated in several of the activities in the course. Barett highlights Black and Williams research findings when she underscores the importance of formative assessment as part of a portfolio assessment. The research conducted in the United Kingdom (Black & William, 1998) provides firm evidence that "formative assessment is an essential component of classroom work and that its development can raise standards of achievement" more effectively than any other strategy (Barrett, 2006). Formative assessment is emphasized in the course by requiring students to pass four assessment assignments, one at the end of each modul, to fully complete the course. Whitelock excel in her article E-assessment: developing new dialogues for the digital age a paper that examines the role of web-based portfolio assessment by Chang and Tseng. Their findings suggest that the most significant effects on student performance when using the e- portfolio were upon self-assessment, reflection, goal setting, problem solving, data gathering and peer interaction (Whitelock, 2009). All these skills are represented in the course which Mason et al. has done their survey. Dysthe and Engelsen says in their article that quality criteria are important for students in order to self-assess, but these criteria also needs to be negotiated in order for students to make them their own. In a way, the students taking the course has been exposed for this, by letting them choose from 100 learning objects, the students get participation in the content of the criteria, these choices could be made depending on their personal interests, job relevance and/or time and inclination. Dysthe also point out that portfolios need to count more directly towards the grade in order to make it worthwhile to invest time and effort in them (Dysthe & Engelsen, 2004). Here I feel that the program is succeeding. Reflections on how to apply the idea of this paper in a relevant educational context The article is relevant for my daily work as a teacher; in the context of portfolio work in Fronter (I will explore Google Docs as a portfolio later this year). I tried out portfolio as a method in one of my science classes, with encouraging result and motivational feedback from 6
the pupils (See attachment for assignment/result). However, I experienced lots of questions and a bit confusion among some students in the initial phase. By reading this article, it inspired me to want to combine e-portfolio and learning object, not very different from the way we work in the e-assessment course. I will put effort and time in the design of learning objects, they should not be too abstract, nor too narrow or be restricted to lower-level cognitive skills. And to follow up on Dysthe s findings, let the pupils take part in the composition of the criteria given from the learning objects. This should be developed so that students understand what is expected of them, and so that they can use feedback to see where they need to strengthen their performance. Personal comments on strengths and weaknesses The strength of the presentation of this article is the connection to pre known theories, this is supported by peers and teachers in the discussion forum in Fronter. It shows a wide range of connections to several of the other articles that constitute the curriculum of this course. Regarding weaknesses, both peers and teachers pointed out that the reflections on how to apply the idea of this paper in a relevant educational context, could be elaborated. They point in the direction of my own experience with the subject. I have added the teaching plan for this given task, to follow up on my description of the experience in that section, the assignment was conducted in 2012. I also added a delivered film product from a group of students, just to show an example on how the task was interpreted and conducted. This film is published with permission from the students who produced the film. See attachment for both teaching plan and movie. 7
References Dysthe, O., & Engelsen, K. (2004). Portfolios and assessment in teacher education in Norway: a theory based discussion of different models in two sites. Assessment & Evaluation in Higher Education, 29(2), 239-258. doi: 10.1080/0260293042000188500 Erstad, O. (2009). Changing Assessment Practices and the Role of IT. In J. Voogt & G. Knezek (Eds.), International Handbook of Information Technology in Primary and Secondary Education (Vol. 20, pp. 181-194): Springer US. Mason, R., Pegler, C., & Weller, M. (2004). E-portfolios: an assessment tool for online courses. British Journal of Educational Technology, 35(6), 717-727. doi: 10.1111/j.1467-8535.2004.00429.x Whitelock, D. (2009). Editorial: e-assessment: developing new dialogues for the digital age. British Journal of Educational Technology, 40(2), 199-202. doi: 10.1111/j.1467-8535.2008.00932.x Barrett, H. C. (2002). Introduction to Electronic Assessment Portfolios. http://www.helenbarrett.com/ali/intro.pdf Barrett, H. C. (2006) Portfolios. http://www.helenbarrett.com/portfolios/connectednewsletter-final.pdf 8
Attachment Assignment and link to one of the end products (Published with permission from the students who produced the film). Filmprosjekt i naturfag, Arv og miljø Charles Darwin (1809-1882) Tema: Evolusjonsteori og Charles Darwin Dere skal lage en film ved hjelp av Windows Live Movie Maker på skolens pc-er eller dere kan bruke egne pc-er med egne programmer. Dere har da ansvar for disse selv, og må ha de med til naturfagtimene i denne perioden. Produktet skal leveres på Fronter som nevnt nedenfor. Filmens tema skal være evolusjonsteori og Charles Darwin. Det er ingen andre kriterier enn dette, hvordan dere griper oppgaven an bestemmer dere på gruppen. Alle naturfagstimene i uke 42, 43, 44 og 45 er satt av til dette. - Innen torsdag i uke 43 skal dere levere et midlertidig produkt til veiledning, dette skal ligge i Individuelle mapper (Se under). - Leveringsfrist for ferdig produkt er torsdag i uke 45, innleveringsmappen stenger kl. 23.59. I
Dette er en gruppe oppgave, og gruppene skal ikke overskride 3 elever. Dere står fritt til å velge hvem dere vil jobbe med. Lagring av film underveis i prosjektet: Lagre prosjekt. Filmfilen lagres på Fronter i Individuelle mapper i venstre meny. (En på gruppen må gjøre dette). Husk at eventuelle bilder og lyd også må lastes opp her. Kan være lurt å opprette en mappe som heter Naturfag i den Individuelle mappen, og legge alt her. Hvis denne personen ikke er tilstede en time, så ordner Kristoffer tilgang til denne mappen for de andre. Lagring av sluttprodukt: Lagre film Det er denne filen du laster opp i innlevering til slutt. Innlevering: Gjøres på Fronter i Innlevering i venstre meny. Lastes opp i mappen: Arv og miljø:filmprosjekt. Ved gruppeinnlevering: husk å hak av for de du skal levere for når du laster opp oppgaven. Filmen skal fremføres fredag 26.10, 5. time. (Auditoriet). Kristoffer. End product Link til film: Historien om Charles Darwin og evolusjonsteorien II