Piloting VLE communication tools in a Level 1 course Karen Kear & John Woodthorpe Communication & Systems Department, Faculty of Mathematics, Computing & Technology and Centre for the Open Learning of Mathematics, Science, Computing and Technology together with T175 Associate Lecturers: Sue Hawthorne, Sarah Horrigan, Mike Hutchison, Peter Langford, Elizabeth Morrow, Sandy Robertson, Diane Saxon, Janet Seaton & Julie Wright Introduction The Level 1 course T175 Networked Living: exploring information and communication technologies makes considerable use of online tools to support students learning. In particular, T175 adopts a blended approach to tutorial provision, with many of the tutorials consisting of pre-designed activities which are carried out by students online. As part of the course content, T175 teaches students about a number of different computer-mediated communication technologies, including instant messaging, blogs and wikis. The arrival of the OU VLE now provides opportunities for students to gain practical experience of these technologies. A group of T175 Associate Lecturers, together with members of the course team, has recently carried out a pilot project to use some of the OU VLE tools with their students. In this project, which was funded by the VLE team and Student Services, the VLE wiki and blog were offered to students as alternatives to FirstClass for general communication, and specifically for taking part in the online tutorials. The tutorials were not redesigned for the project, but ALs provided modified instructions and resources. In addition, the blog was offered as an alternative to the course online learning journal. The latter facility, which is available from the course website, is designed to encourage students to reflect on their learning. Students use of the VLE tools was evaluated using an online questionnaire at the end of the course. This provided both quantitative and qualitative data on students reactions to the VLE tools. Comparisons could be made between using the VLE tools and using FirstClass for the same set of activities: the online tutorials. Similarly, an assessment could be made of the potential of blogs for learning journals. 1
Research approach and activities Nine T175 Associate Lecturers (ALs) took part in the project, which took the form of action research, with the ALs acting as researchers as well as teachers. The ALs aimed to use a selection of the VLE tools - blogs, wikis and instant messaging - to support their work with students on the 2007B presentation of the course, which started in February and ended in October 2007. Some of the tools were more advanced than others at the outset of the project. One problem was the lack of access control, which meant that activities could not easily be contained within a tutor group, or even within the course cohort. This was a particular problem with the blog and instant messaging tools. Because there were additional problems with the instant messaging tool at the time of the project, it did not form part of the evaluation. The nine tutors who took part in the project had 161 students between them at the start of the course, and 111 at the end. Most of the participating tutors offered the VLE blog to their students as a possible replacement for the course online learning journal facility. Most of the tutors offered the VLE wiki (rather than the blog), as an alternative to the FirstClass forum for the Block 3 online tutorial. The tutors created template pages within the wiki for their students to use. Several tutors also offered the wiki for the Block 4 online tutorial. For the Block 3 online tutorial, students worked in pairs, allocated by the tutor. Each student chose one of the block learning outcomes and wrote an entry in the wiki, explaining how he/she had fulfilled this learning outcome. Students provided constructive feedback on their partners entries, by adding a further contribution to the wiki. Each student then used the feedback to improve their initial entry, which they submitted as part of the assignment for the block. For the Block 4 online tutorial, the student group evaluated a selection of three websites. Each student wrote an evaluation of one site and posted this to the wiki. As students carried out this task, the wiki built up into a set of evaluations of the three websites. Using the wiki as a resource, students then wrote a comparative evaluation of two of the websites and submitted this as part of their block assignment. At the end of the course, an online questionnaire was made available to all students who took part in the project, and 54 students completed the questionnaire. In terms of the number of students who took part in the pilot and completed the course (111), this is a response rate of 0.49. The questionnaire contained two types of question: Closed questions, where students selected one answer, or several options, from a list; Open questions, where students typed in their views. 2
Quantitative data from the questionnaire The quantitative data from the questionnaire was summarised, and analysed. The most significant findings are presented below. Demographic and background data Figure 1 shows the gender and age distributions of respondents to the questionnaire. There was a bias in the respondents towards males, and most respondents were in the 21-40 age band. These findings largely reflect the gender and age distributions for the course cohort as a whole. Age and gender of sample 20 18 Number of respondents 16 14 12 10 8 6 4 2 0 20 years old or less 21-30 31-40 41-50 51-60 61-70 (blank) Age Male I'd rather not say Female Figure 1. Age and gender distributions of questionnaire respondents Table 1 shows the distribution of educational experience in the sample. The various educational levels are relatively evenly spread, though with slightly more respondents in the further education category. Table 1. Educational experience of respondents Education Further education (e.g. a 6th form college) 18 Secondary School 12 Traditional Higher Education (e.g. University) 12 Vocational higher education (e.g. community college) 12 3
Respondents were asked about their internet connection speeds. It was found that nearly half of the sample were on fairly slow Internet connections (less than 1 Mbps), with about 1 in 20 respondents on dial-up. Younger students tended to have faster internet connections. Respondents were also asked how they accessed the FirstClass conferencing system (the standard communication tool for the course). Only 6 of the respondents used the FirstClass web interface exclusively, compared to 24 using the FirstClass client exclusively, and 20 using a mixture of the two methods. Use and perceptions of VLE tools Table 2 gives a breakdown of respondents VLE use for the two online tutorials. For the Block 3 tutorial, 44 respondents used the VLE. For the Block 4 tutorial, 23 did so, with all but one of these students having also used the VLE for Block 3. Several ALs chose not to offer the VLE for the Block 4 tutorial, either because they wished to compare use of the VLE in Block 3 with use of FirstClass in Block 4, or because they found the VLE tools too time-consuming. For each online tutorial, the questionnaire asked students whether they were offered the VLE, and whether they had used it. Taking into account only those respondents to whom the VLE was offered, Table 2 shows a substantial drop in the percentage of students using the VLE in Block 4, compared with Block 3. One interpretation is that students were reluctant to continue using the VLE after an initial exposure to it. Table 2. Proportions of respondents who chose to use the VLE Block 3 tutorial Block 4 tutorial Respondents who were offered the VLE for the tutorial 52 41 Respondents who used the VLE for the tutorial 44 23 Percentage use 85% 56% Students who had used the VLE for the Block 3 or Block 4 online tutorials were asked to express their views about the experience, by agreeing or disagreeing with a number of statements. Table 3 shows the responses to each statement for the two online tutorials. In each case, the figures are given as a percentage of the number of students who used the VLE for that tutorial. The smaller numbers participating in the Block 4 tutorial (n=23) mean that the results are less robust than for Block 3 (n=44). 4
Table 3. Respondents views on using the VLE tools for the online tutorials Block 3 Tutorial Block 4 Tutorial I enjoyed using the wiki or blog 73% 57% I found it confusing having other people edit what I had written 9% 0% I felt irritated when other people changed what I had written 16% 17% I did not feel happy about editing other people's work 30% 9% I found adding new material to the wiki or blog easy 70% 48% I found it difficult to add new material to the wiki or blog 16% 22% I never really mastered adding new material to the wiki or blog 14% 9% I prefer to use the tutor group conference in First Class to collaborate with others in my group 52% 52% Wikis and blogs are good ideas, but the current OU systems are difficult to use 50% 43% I don't think I could ever be comfortable using a wiki 9% 9% I don't think I could ever be comfortable using a blog 14% 9% As Table 3 shows, for the Block 3 tutorial a large majority of respondents enjoyed using the VLE. For the Block 4 tutorial this decreased to just over half, possibly because the novelty value had reduced. In Block 3 a small proportion of students were confused or irritated when others edited what they had written. In Block 4 the confusion disappeared, but the degree of irritation remained. In Block 3 almost a third of the respondents were unhappy about editing other people s work, but by Block 4 this concern had reduced. The remaining data in Table 3 suggest that some students found the VLE tools difficult to use, and these difficulties did not reduce with time. For both online tutorials, just over half of the respondents indicated that they preferred using FirstClass for collaboration. For the Block 3 online tutorial, where there were larger numbers of responses, the answers to the statement I enjoyed using the wiki or blog were disaggregated by age, gender and connection speed. Although there were no gender or age effects, students with faster connections were more likely to say that they liked the wiki or blog than those with slower connections. Students were asked a number of questions comparing their experience of using each of the VLE tools - wiki and blog - with their experience of FirstClass. Figure 2 illustrates students responses to questions comparing the wiki with FirstClass, in terms of three aspects of usability: time to learn; time to edit and post contributions; ease of use. 5
30 Number of responses 25 20 15 10 5 Time to learn Time to edit Ease of use 0 wiki much worse wiki a bit worse About the same wiki a bit better wiki much better Not sure Figure 2. Respondents perceptions of the wiki in comparison with FirstClass In Figure 2, more responses lie to the left of the about the same mark, suggesting that students perceptions of the wiki were less favourable than their perceptions of FirstClass. The overall impression is that the wiki took longer to learn, and was slower and more difficult to use than FirstClass. Figure 3 illustrates students responses to similar questions comparing the VLE blog with FirstClass. 30 Number of responses 25 20 15 10 5 Time to learn Time to edit Ease of use 0 blog much worse blog a bit worse About the same blog a bit better blog much better Not sure Figure 3. Respondents perceptions of the blog in comparison with FirstClass 6
As Figure 3 shows, far fewer students had used the blog, but the perceptions of those who had used it were, again, less favourable when compared with their perceptions of FirstClass. These findings in relation to the wiki and blog may partly be caused by students familiarity with FirstClass and unfamiliarity with the VLE tools. However, such large differences strongly suggest that, at the time of the project, usability aspects of the VLE tools needed to be addressed. Qualitative data from the questionnaire Qualitative data was obtained from students via open questions in the questionnaire. These asked students for : their comments on using the VLE for the online tutorials; their thoughts on whether they would use the blog or wiki again. Thirty-four students made 70 comments in response to the questions, and these responses fall naturally into the categories of wiki and blog. The wiki There was a fairly wide set of views about the wiki. The main areas of comment were as follows. This is a good idea Several students gave positive comments on using the wiki. These students reported finding the experience interesting and enjoyable. Yes I d use it again. It was good fun to edit other people s work and contribute to a group produced document. Other positive comments related to organising and presenting ideas, and communicating them to others. Yes I would use the wiki again as I thought it was a really good way to get ideas and thoughts organised. Usability problems Students who displayed a negative view of the wiki primarily did so because of problems related to usability or lack of functionality. Yes I d use it again, but it s not an easy system to use Several students said that they could not do what they wanted, presumably either because they could not find a way, or because the capability was not present. It was difficult formating text as there seemed to be a default type / size. Separation of messages needs to be better. Placing images is difficult - I'm convinced some buttons were missing. Help files needs improving. 7
Social and personal discomfort At the time of use, the wiki was open for read and update access to anyone with an OU user- ID, and this caused students some concern. Several students were not happy that others could edit their work and that they could edit the work of others. I would not use it, it is not secure enough for me. I felt it was too open allowing anyone to modify it without my knowledge Don t like having the option of editing other people s work why?? Participation and collaborative working There were a number of comments related to lack of participation in the on-line group activities. Although this issue applies to any tool used collaboratively, perceived difficulties in using the wiki may have discouraged some students from participating. I found the Wiki very impersonal and tended not to use it for anything other than the tutorial. I felt uncomfortable using it and with such an open format I feel it is too open. However, some students commented on the value of a wiki as a tool for creative collaboration. Wikis are something that has good potential. They make it easy to collaborate jointly on documents which has always been a bit of a logistical nightmare. Further requirements and functionality A number of students pointed out specific facilities which were missing from the wiki, and which they felt were needed. In particular, students wanted to know when a new contribution had been added, and who had viewed contributions. No immediate way of knowing if new messages are present, without opening the Wiki up and searching all the headings - time consuming. No history - can't see who is present, who has read the messages - very frustrating. These facilities were familiar to students from their use of FirstClass. I preferred the first class tutor group conference to collaborate with others in my group, mainly because it was easier to access and quicker to use, but mostly because in First Class it is very easy to see any new messages people have added. It is likely that prior use of FirstClass influenced students expectations of the VLE communication tools. Some students may have preferred FirstClass because they were already familiar and comfortable with it. However, students comments suggest that, at the time of the project, FirstClass was genuinely easier to use, and had better functionality, than the VLE wiki. 8
The blog Students comments about using the blog were mixed. There were some positive reactions to the idea, but students also expressed concerns regarding privacy. Blogs as communication tools Some students were enthusiastic about experiencing a new communication technology. It was the first time I had used a blog and I found it a pleasant and easy experience that I would happily use again. Others were somewhat less positive, seeing few benefits of the blog compared with using FirstClass. I did not use the blog, as I was happy with using First Class, however I think that the OU should continue to offer and develop this option. Blogs for learning journals Many of the comments related to the possibility of using the blog as a learning journal. Students were concerned about the lack of privacy of this option. My personal Journal was done in Word on my computer and therefore exactly that personal. The general view seemed to be that, though blogs were of value for certain purposes, they were not suited to keeping a learning journal. No, I think "blogging" is better suited for keeping in touch with mates or subjects / people that you are interested in. I believe that students should be aware of such technologies, but not sure how relevant they are for journals. Discussion and conclusions The findings of the project show that T175 students valued the opportunity to try out the new communication tools in the VLE. Most students enjoyed the experience, and felt that the exercise was well suited to the course. However there were significant usability problems with the tools, and these were frustrating to students and tutors. The VLE tools were not easy to learn or to use. Another issue which was highlighted in this project was privacy. At the time of the project, the VLE tools had few facilities for restricting access. This meant that students contributions could not easily be kept within their tutor group or, in the case of the blog, even within the course. This caused concern to both to students and tutors. When students are experimenting with new communication facilities, and learning new concepts, it is reassuring for them know that they are in an enclosed environment. In relation to the wiki, some students were concerned about the prospect of editing others work, and having other students edit their own. This issue is not entirely restricted to wikis; students also express concern when asked to critique each other s work in other contexts. 9
However, the facility to edit others work is novel to most OU students, who are more familiar with interacting via forums, where each person s postings cannot be changed. With time, students became more comfortable with the idea, but this issue needs careful consideration when contributions to a wiki form part of the course assessment. An issue to note, in relation to the research reported here, is that the online tutorials were designed for use with FirstClass, rather than with wikis or blogs. Translating these activities for use with VLE tools may not give an entirely fair assessment of the tools. Nevertheless, the project has provided an evaluation of the VLE tools as compared with FirstClass, and has thus contributed to an improved understanding of their different affordances. It has shed light on students perceptions of the VLE tools, has highlighted issues that need to be addressed, and has explored how the tools could be developed for the future. 10