eportfolio Trials in Three Systems: Training Requirements for Campus System Administrators, Faculty, and Students Mary Bold, Ph.D., CFLE, Associate Professor, Texas Woman s University Corin Walker, M.S., Instructional Coordinator, Texas Woman s University Lillian Chenoweth, Ph.D., Professor, Texas Woman s University At Texas Woman s University (TWU), faculty and students required training for a 2005 trial use of Blackboard Content Management System for eportfolio creation and 2006 trials of TrueOutcomes and TaskStream. These three platforms for eportfolios are among about a dozen products that currently count universities, community colleges, and K-12 schools as customers. Like many small universities today, TWU struggles with the challenge of software adoption as products (and pricing structures) change annually and learning communities across campus differ in their preferences for software. Without a strong history of making campus-wide adoptions, programs seeking eportfolio software relied on faculty and students willingness to experiment in order to learn what software features would be valuable. Exploring several platforms in short trials allowed faculty to see how eportfolio systems performed in real-world use but challenged everyone in having to learn three systems across four semesters. The program that utilized all three systems for trials is a wholly online Master s degree in Family Studies. Since the program s inception in 2002, graduates had submitted on CD-ROM an electronic portfolio of selected course work products as their final project in the M.S. program. Thus, the program already had a set of portfolio Page 1 of 10
guidelines and the faculty was accustomed to evaluating students exit projects through computer display. Also, students who would be asked to experiment in the trial platforms were working professionals who had sought a Master s program that is 100% online. A fair assumption could be made that these students would welcome the opportunity to use and rate an alternative technology for the project. This was the case; almost all graduates every semester agreed to work in the trial platform. Uses of eportfolio Electronic portfolios are displayed via computer. They are typically uploaded to the Internet but may also be affixed to CD-ROM or DVD-ROM. The files comprising a portfolio may be a collection of works as they were created or may include revisions of work; almost always, the portfolio includes reflection on the body of work. If the portfolio was created to document or assess, it also includes analysis and evaluation. Well established in the arts since the 1980s, portfolios today are becoming entrenched as exit requirements by colleges of education (Lorenzo & Ittelson, 2005a). The term eportfolio has come to refer to an electronic portfolio that is created through web-based software and displayed to others on the Internet. As eportfolio platforms are offered for student use, educators fairly ask if the electronic version of a portfolio is superior to a hard-copy or physical one. The differences lie in the ability to duplicate and distribute the product with little cost or effort, the capacity for multimedia files, and portability that permits work on a number of platforms or software products. In terms of the compilation process, the eportfolio saves students time; in terms of evaluation, an eportfolio platform permits electronic scoring and reporting. Page 2 of 10
Widespread adoption of eportfolios can be dated to the mid-1990s. As commercial platforms have proliferated (since year 2000), adoption has been an easier and easier decision. From a data collection perspective, the most efficient eportfolio would be fully integrated with a learning management system (LMS) and institutional data content management system. Functions of eportfolios include documentation, planning, evaluation, jobseeking; and uses include student presentations, program review, and institutional effectiveness (Lorenzo & Ittelson, 2005a). While universities and schools have routinely made self-study and documented it, the use of electronic records to represent self-study is new. A driving force may be that since 2004 SACS has increasingly relied on an institution s webpage or internal eportfolio for documentation under review (Lorenzo & Ittelson, 2005b). Thus, at the same time that schools are adopting eportfolio platforms and standards for their students use, administrators and faculty are establishing processes that they are likely to use for their programmatic documentation, including faculty credential and performance reports. For students, eportfolio options cannot rival social software but especially undergraduates respond positively to the opportunity for self-expression as they create a web-based representation of their studies and career goals. For academic programs, eportfolio permits the tracking of students development over time. At the University of San Diego, advanced programs use a model for student checks at admission, midway through program, exit or graduation, and follow-up post-graduation. In addition to working on a commercial eportfolio platform, the school uses Dreamweaver and Access to form its database management system (Ammer, Getz, & Hubbard, 2005). The extent Page 3 of 10
to which student works are supported and for how long is still being debated, even at schools that currently require eportfolio creation by all registered students. In short, questions of file/portfolio sizes and length of time of storage/access are not always answered before school-wide initiatives begin. For faculty, eportfolio use is typically first seen as a teaching tool and only later adopted as a means to document faculty s own performance areas (if the eportfolio is to be used for annual evaluation or for tenure/promotion application) and credentials (for institutional reporting for accreditation, for example). In some disciplines, a constructivist model describes faculty eportfolios: the electronic collection of works that not only increases over time but also changes qualitatively as the faculty member continues to develop as scholar. For administrators, eportfolio can serve as a data source of aggregated assessments of student works, faculty credentials, and program tracking records of student progress (especially as it relates to retention and graduation). National standard sets are included in leading eportfolio platforms, increasing the value of eportfolios to meet accreditation requirements and to serve as documentation in accreditation reports. eportfolio can support several uses across a campus; however, a single platform may not be adequate for an institution s full reporting purposes. At best, an eportfolio platform contributes to a larger system of inputs. Investigating eportfolio Systems TWU s Family Studies program first participated in a two-semester trial of Blackboard s eportfolio system. As the second semester trial drew to a close, the program conducted a review of leading stand-alone eportfolio systems (i.e., not part of Page 4 of 10
a learning management system or LMS, like Blackboard). Through web and telephone research with vendors, a list of platforms was created for remote desktop demonstrations and from those events, TrueOutcomes was selected for the second trial. Finally, TaskStream was selected for the third trial. As eportfolio systems were reviewed, a list of concerns was developed. Although not all concerns are managed at the academic program level, faculty are nevertheless interested in how individual platforms perform, realizing that the companies performance and capacity will be compared by technical staff. Cost to student, to university Server location (hosted or on campus) Accessibility (e.g., Section 508) Customization (templates), Branding Rubric maker Standards sets (for national and state accreditations) Help Desk (cost, hours, provision by company, provision by university) Consultancy service for educators Length of access - by student to materials, to webspace Portability - student's ability to download, burn to CD Archive options (e.g., to support studies of institutional effectiveness) Practical limits of long-term storage Security of student products (regardless of location of server) Copyright of student material Copyright of educator material Page 5 of 10
Copyright of institution material Training, support for educators and students Cost, time of sysadmin on campus Training Requirements In the case of Blackboard s eportfolio, technical staff and selected faculty were trained by a Blackboard employee in a day-long workshop on the LMS s Content Management System (a required component for the eportfolio). For TrueOutcomes, the company s founder and CEO trained faculty through telephone conferencing. For TaskStream, an introductory 2-hour training was conducted in person for two faculty members and subsequent training was through telephone conferencing. In all cases, the companies representatives were responsive to faculty members questions and concerns. In all cases, the trainers were knowledgeable about their products and also about educational issues. TrueOutcomes and TaskStream personnel were especially well grounded in assessment techniques, including the pragmatic matter or creating and integrating rubrics to support electronic scoring of student products.. To introduce the platforms to student users, training processes and materials were developed at the program level. For Blackboard eportfolio across two semesters, faculty members and a graduate student conducted in-person workshops on campus for students within driving range of TWU. (A few students at greater distance also used the system, without training in workshops.) For TrueOutcomes and TaskStream trials (each one lasting one semester), no in-person workshops were utilized. Faculty members worked with students through email and web-based discussion boards. Also, for these platforms, program faculty created how-to guides and illustrated flyers for student use. Page 6 of 10
These supports were important to students as they entered a system and made initial efforts at submitting portfolio elements. Examples of the guides are available at http://www.famsci.com/portfolio/report. Training Challenges Through running three successive trials, a certain efficiency developed so that the last trial was the least challenging. The Family Studies faculty began to think in terms of next time, and consequently made better start-up decisions. This gain in efficiency also reflected the greatest training challenge: seeking an investment of faculty time and energy was difficult when everyone knew that the effort would not necessarily result in a permanent platform adoption. A philosophy developed among the faculty: that they should be prepared for many such changes, although hopefully not so frequent, as software and services develop in future. Some of the improvements in faculty training across the three trials were: (a) increased number of fake student accounts so that many people could go into the system and create materials and then a group of products could be evaluated, (b) one or two faculty members serving as screeners of students work so that the full faculty team of six could make efficient review of many products at one time, and (c) how-to guides with (color) screen shots of faculty view of the platform. The most important improvements serving students were: (a) creation of a short Practice program where they could receive feedback from faculty or assistants on how their files looked, and (b) location of a Q&A discussion board within the eportfolio platform. A major challenge in training was the time and cost of creating customized howto s and guides for students. Although each platform had some sort of support (manual Page 7 of 10
or help desk or both), students nevertheless required materials created locally. The Family Studies guides integrated existing portfolio guidelines and also used screen shots displaying the TWU brand. Most helpful was the local guides explanation of each system s unique vocabulary, which challenged both students and faculty. Some words and functions were described in terms of, and compared to, Blackboard LMS words and functions because TWU is a Blackboard school and thus all of the online students were familiar with that vocabulary. Challenges that defied solution were those that routinely plague us in technology use. Acceptance and adoption frequently depend on (a) a personal change model; (b) the institution s change model; (c) one s status on the curve described by Everett Rogers (2003) theory of diffusion (innovator, early adopter, and so forth); (d) and, perhaps most deadly, WYKIWYL. The acronym stands for What you know is what you like, and simply means that once a person has made investment of time in learning software, he or she is likely to consider that software superior to any competing product. Experiences and Observations 1. eportfolio platforms are in various stages of development and they typically make upgrades every year. The best platforms allow organization by course and then flexibly aggregate and report data by course, by learning outcome, by program outcome, and with demographic information. 2. The Family Studies program faculty came to terms with the fact that eportfolio updates, upgrades, and whole platform moves will be the norm. Thus, focus should be Page 8 of 10
on the pedagogical / andragogical goals; technology tools should be learned quickly and completely, but should not drive the processes. 3. eportfolio use is most effective when phased in. Early adopters can help establish best practices and create guides for later adopters. A program already using a portfolio will have a starting point; a program that does not use portfolios may choose to use the platform for data collection on course-embedded assessments for institutional effectiveness reports or similar program review. 4. An academic program should schedule a minimum of one month for even a partial ramp-up with a new platform. The month's set-up could produce 2-3 rubrics for use in selected courses by several faculty members, or several rubrics for use in an existing portfolio for a limited number of graduating students. Longer preparation would be needed to institute an eportfolio for the first time. 5. If the eportfolio is organized to reflect course-embedded assessments, it can support several functions that will benefit retention, learning outcomes, and academic review at both the program and institution levels: (a) identify 1-4 assessments to analyze for program evaluation and annual institutional effectiveness reports, (b) identify products to be collected for use in an exit portfolio, (c) track individual student progress from first semester to last, (d) aggregate data from multiple rubrics (across multiple products) to report on student learning outcomes, (e) aggregate data to report on whole program outcomes, (f) create base for student's professional portfolio for job search or professional development. Page 9 of 10
References Ammer, J., Getz, C., Hubbard, L. (2005). Comprehensive e-portfolio NCATE assessment system: Rethinking learning & teaching. Paper presented at the annual Syllabus 2005 Conference, Los Angeles. Lorenzo, G., & Ittelson, J. (2005a). An overview of e-portfolios. EDUCAUSE Learning Initiative, ELI Paper 1:2005. Lorenzo, G., & Ittelson, J. (2005b). An overview of institutional e-portfolios. EDUCAUSE Learning Initiative, ELI Paper 2:2005. Rogers, Everett M. (2003). Diffusion of innovations. New York: The Free Press. Page 10 of 10