305 Synchronous Small- Group Collaboration: Promoting Higher- Order Thinking Jennifer Hofmann Principal e-learning Consultant, InSync Training WWW.eLearningGuild.com
Research Results: Synchronous Small-Group Collaboration: Promoting Higher-Order Thinking in the Virtual Classroom Using Breakout Rooms The complete research study (98 pages), including techniques and references, can be downloaded from: http://www.insynctraining.com/pages/promotinghigherorderthinkinginthevirtualclassr oomusingbreakoutrooms.pdf Slides from this presentation can be downloaded from: http://www.insynctraining.com/pages/promotinghigherorderthinkinginthevirtualclassr oomusingbreakoutrooms-slides.pdf Introduction The problem addressed in this research study was that instruction delivered via a live, online delivery format tends to be very lecture oriented and generally does not utilize small group interaction techniques for practice and content reinforcement. As a result, participants had less of an opportunity to achieve higher-order thinking with regards to the subject matter. This goal of this research study s intervention was to equip online instructors and course developers with the exposure, instruction, practice, and tools required to incorporate small group interaction techniques into virtual classroom programs for the purpose of promoting higher order learning. Under the direction of the researcher, participants engaged a series of online interventions in order to achieve the program outcomes. The anticipated outcomes as projected prior to the study implementation, with results, are listed below. Results The problem addressed in this research study is that instruction delivered via a live, online delivery format tends to be very lecture oriented and generally does not
utilize small group interaction techniques for practice and content reinforcement, severely reducing the opportunity for participants to achieve higher-order thinking with regards to the subject matter. This goal of this research study s intervention was to equip online instructors and course developers with the exposure, instruction, practice, and tools they needed to incorporate small group interaction techniques into virtual classroom programs for the purpose of promoting higher order learning. Under the direction of the researcher, participants engaged in a series of online interventions in order to achieve the program outcomes. Outcome 1: By the end of the implementation period, participants will be able to use the breakout room feature available in the WebEx Training Center Virtual Classroom. The outcome was partially achieved. Participants attended a one-hour session to demonstrate how the breakout room feature, which allows live online groups to be broken into smaller groups for the purpose of collaboration, works. The session was delivered by an instructor who worked for the software vendor for over five years in the role of software instruction, and so had the appropriate expertise and experience to provide this instruction. The instruction was supplemented by software documentation provided by the vendor. Participants had the opportunity to attend virtual office hour sessions to get additional instruction and guided practice in setting up and using this feature no participants took advantage of these sessions. Of the two teams that eventually presented and facilitated their designs, one managed the technical aspects completely on their own. This team had practiced,
independently, at least twice prior to presenting, and indicated aloud that the practice time made it possible for them to be successful. The second presenting team required assistance loading content into the various rooms and demonstrated success after that. This second group indicated that they practiced delivering content, but not technically setting up the rooms. The third team was not able to demonstrate expertise due to complete server failure of the virtual classroom the program was not able to be rescheduled because of the end of year holiday season. Outcome 2: By the end of the implementation period, participants would be able to design effective small group collaborations utilizing the breakout room capability of the virtual classroom. The outcome was achieved. The three groups all successfully designed collaborative activities that encouraged interaction and involvement among small groups. Groups were required to create slides, facilitator materials, and student materials to support their activity and to facilitate the activity for the rest of the study participants. (See Appendix III for the final project description and requirements.) Each group worked together to create materials that met the requirements and support the concept that higherorder thinking can be achieved in the online environment if due consideration was given to designing the activity. Participants completed a survey designed to inform the researcher of the participants evaluation of their personal design abilities at the end of the study. The survey results revealed that 100% of respondents understood how to design synchronous activities to promote both lower- and higher-order thinking skills. 100% of respondents also believed that small group activities can be effectively conducted in the synchronous
classroom. 80% felt that synchronous collaboration could meet or exceed what could be achieved in a traditional environment. (See Table 1.) Table 1 Breakout Room Research Final Participant Survey Raw Data SA = Strongly Agree A=Somewhat Agree D=Do Not Agree N/O=No Opinion Questionnaire Items SA A D N/0 6a I understand how to design activities for the synchronous classroom that promote lower order thinking skills. 8 2 6b 6c 6d I understand how to design activities for the synchronous classroom that promote higher order thinking skills. I believe that small group activities can be effectively conducted in the synchronous classroom I believe that the quality of collaboration in the synchronous classroom can meet or exceed the quality of collaboration in the traditional classroom. 3 7 7 3 4 4 1 1 6e I am able to design effective breakout room activities. 3 6 1 When providing commentary associated with the questions in survey questionnaire, participants noted the following issues associated with their being able to design higher-order activities. Participants reported difficulty initiating breakout room activities when a small group leader was not designated. Since time was spent determining how they would work logistically, less time was available to thoroughly complete the activities and achieve high-order results. Not all participants had access to breakout room technologies within the virtual classroom available at their workplace. This resulted in frustration as they tried to work around their technology tools to find solutions for small group collaboration. While most felt that small group activities can be effectively conducted in the synchronous classroom, participants felt strongly that the technology should be as easy as possible to use to maximize the results. For example, a voice system that
would transfer participants from the large group teleconference to a small group teleconference without participants having to redial would be optimal. Outcome 3: By the end of the implementation period, participants would be able to facilitate effective small group collaborations utilizing the breakout room capability of the virtual classroom. The outcome was partially achieved. After the initial live instructional session, the researcher identified the need to provide time for the groups to work together during the planned instructional period. This reduced the amount of time available to deliver and discuss new content by approximately two hours, necessitating the need to move this topic area to an asynchronous reading assignment. Participants based facilitation strategies on modeling by the facilitation staff and past experience as facilitators in various roles. Final survey results indicated that 30% of participants strongly agree and 70% of participants somewhat agree with the statement: I am able to facilitate effective breakout room activities. Outcome 4: By the end of the implementation period participants would have worked together to design a 20-30 minute breakout room activity, including participant materials, instructor notes, and slides, as appropriate. This outcome was achieved (see Outcome 2 above), but with surprising results as to how the groups worked together to create the design. It is apparent that the concept of online collaboration was being tested on several levels. Participants were being asked to interact and collaborate with a group of geographically distributed individuals who were hitherto unknown to each other. The groups faced challenges of scheduling, participation, and organization but to different extents. One group took up the gauntlet
immediately, scheduling independent meetings, determining group process, and mutually determining a direction. At the other end of the spectrum, another group expended a significant amount of energy during asynchronous discussions trying to accomplish similar goals, but were not able to achieve consensus due to what its participants felt was a lack of clarity in the project instructions and expected outcomes.. When asked about the small group work in the final survey, responses included: Our group collaboration was a real struggle. We had people in multiple time zones, and I admit that with all of us working, it was difficult to get a consensus on anything. I found it very difficult to communicate. There were obviously different styles in the group and the culture gulf was a rather large one to breach via virtual communication and time constraints. Involvement of the participants only started late when the pressure of the upcoming delivery brought up the priority over personal / business tasks. Institutional Review Board (IRB) requirements for anonymity were identified as a strong impediment to team success. Since all classes were recorded, the IRB required that a participant sign into the live sessions using a pseudonym and that we not discuss personal details during the interactions. Participants repeatedly suggested that not being able to know more about group members on a personal level decreased the development of trust. Discussion Overall, the researcher was satisfied with the performance of the participants and the participants were satisfied with the training they received. All of the outcomes were
achieved either partially or completely, and several opportunities for additional research were identified. Once the initial participant group was established, enthusiasm was particularly encouraging reinforcing that the problem that this study was designed to address was something that needed to be solved. Attrition was not due to declining need or interest, but to challenging schedules and changing priorities. (For example, three participants started new jobs after they enrolled in the study, and this change forced them to withdraw.) The length of the study, classes delivered over ten weeks, seemed to have an impact on success, as participants could not foresee how their priorities might change over that period of time. The researcher uncovered challenges that needed to be resolved before the ultimate goal of the study was achieved. Some of the issues can be addressed ahead of time in future interventions others may only be addressed if the format of the study is substantially altered. Challenges impacted the implementation of the study by impacting timelines, frustrating participants, and altering the original study plan. The initial research plan indicated that the study should include four participants from each of the following demographic groups: corporate training, higher education, military and government, and K-12 education. The final group composition was somewhat homogenous and consisted of 15 corporate trainers with three academic professionals. When the researcher initially tried to recruit participants for the study, there were over 120 people interested. After polling this group as to availability, it was eventually possible to schedule ten participants in a mutually agreeable time for the six two-hour
sessions. To find this time, the start of the course of instruction was delayed more than three weeks, pushing the class dates into the busy end-of-year holiday season. On the day the course of instruction was to begin, three people dropped out. The researcher put out a call for more interested parties, and by the end of the day had 18 people able to commit to the majority of the sessions. (Note: The first session had to be repeated to accommodate the last minute participants.) It is interesting to note that people could commit at the last minute but were unwilling to commit a month ahead of time. Many of the participants indicated that 100% attendance was not possible, and were accommodated with recordings of the sessions that were missed. Participants often joined sessions up to15 minutes late or left early because of other work commitments, including teaching virtual classroom sessions. Two participants dropped out on the day of the 3 rd live session, citing work commitments. The result of this was the reassignment of the remaining participants to the other three teams. Two additional participants dropped out after the course of instruction was completed, but prior to completing their projects with their teams. The final session was attended by ten of the original 18 study participants. Participants were not alone in dealing with scheduling issues. After the study had begun, the researcher was forced to schedule several business trips which required that a substitute facilitator be used for one session. After the initial live instructional session, the researcher identified the need to provide time for the groups to work together during the planned instructional period. This reduced the amount of time available to deliver and discuss new content by
approximately two hours, necessitating the need to eliminate the topic concerned with facilitation breakout rooms. Contributing to the feeling of online isolation was the required anonymity of the participant group. Since live sessions were recorded, the Institutional Review Board required that participants utilize pseudonyms to protect their privacy. To support this requirement, the researcher eliminated planned getting to know you activities, lest shared personal information divulged too much information about the participants identities. Ultimately, this was counterproductive to the concept of creating small group collaboration online. A majority of participants felt successful collaboration was delayed for weeks because they were thrown into situations were they needed to depend upon and trust fellow participants about which they knew very little. They complained about the confusion caused when the researcher called upon them using something other than their real names, and the disparity between the names they got to know in virtual classroom participant lists compared to the names they used in their asynchronous email collaboration. Participants actually began a bit of a rebellion, logging onto sessions with their pseudonyms and including their real names in parenthesis, or refusing to use the pseudonyms at all. Several technology issues also impacted the study. One participant was not able to make the virtual classroom software work on her computer, even though she met minimum requirements as stated by the vendor. This resulted in her immediately needing to withdraw from the study. One session was not recorded properly, due to the researcher s operator error. The most notable technical difficulty was a total breakdown
of the virtual classroom technology during the final session, which resulted in the third group not having an opportunity to present and facilitate their designs. Recommendations The research study was designed to promote higher-order thinking in the virtual classroom using breakout rooms. The researcher attempted to achieve this objective by providing instruction to the study group participants on how to design, manage, and deliver breakout room activities. While the individual objectives were achieved, the researcher feels the success of the study was marginal, impeded by unanticipated roadblocks. To improve the results of the study, if repeated, the researcher has the following recommendations: 1. The researcher s experience while delivering this course of instruction has highlighted the need for much more expansive instructions for live, independent, and small group exercises. Activity directions need to be stated aloud, previewed on the screen, and provided in detail in the printed participant materials. While not all participants require this level of reiterated instruction, those that do require it can become completely lost in cyberspace. When participants become confused in a traditional environment, it is relatively easy to get support from nearby colleagues and/or the facilitator. Online, the frustration of isolation can result in temporary disengagement or the feeling of permanent disenfranchisement. 2. If possible, utilize an intact work group with a recognized need for small group online collaboration. Working with colleagues from the same
organization may increase motivation to participate, and reduce the need for introductory activities. Additionally, a group working towards a common goal that ties back to job performance may create an atmosphere where the learning is more meaningful to the individuals of the group. However, the researcher suggests that even when working with an intact work group, that the participants be geographically distributed to encourage online collaboration during the project design. 3. If the study group does not consist of an intact work group, institute some getting-to-know-you activities. Asking individuals without common interest can reduce the opportunity for full commitment. By implementing ice-breaker and other similar activities up front, the facilitator can start to establish an atmosphere of trust. The facilitator may also be able to identify areas of common interest, and use this information to create the design project groups. 4. Provide an orientation to working with virtual teams. Whether or not the study participants belong to an intact work group, the facilitator should institute some instruction on how the teams should work together to achieve virtual success. Suggested topics should include how to create trust, determine roles, establish a work plan, distribute responsibilities, manage conflicting priorities, and meet deadlines. This type of instruction will segue effectively into the topic of online collaboration, as collaboration and teamwork are closely tied and have many of the same requirements. One design idea to meet this objective could be to design a
collaborative exercise around virtual teaming and have the small groups work through that exercise at their first meeting. 5. Schedule instructional sessions closer together to maintain momentum and reduce attrition due to changing priorities. Participants had difficulty anticipating scheduling conflicts so far in advance. The three week delay in program start pushed the implementation right up to the end-of-year holiday season, which exacerbated the problem. 6. Avoid anonymity. Online collaboration seems to require an additional level of trust, and conventions like pseudonyms erect a barrier to that end. 7. Improve materials and activity instructions. Participants suggested adding milestones, potential design topics, and suggested agendas for each workgroup session. Combine all supplemental materials into one document with a thorough table of contents and complete exercise descriptions. Include an example of a collaborative work plan for the small groups plus an example of what they are expected to produce. 8. Use the asynchronous discussion board as a place for the groups to interact asynchronously, but do not require additional homework postings to that area. The homework did not seem to add value to the process, and was viewed as busy-work by the participants. 9. Eliminate facilitation topic/requirement from the curriculum. Design and facilitation are two distinct skills and both require distinct training. By narrowing the scope to just designing small group activities, the number of sessions can be reduced and participants can spend their limited
collaboration time focused on design. Instead of requiring teachbacks, consider having the facilitator run the activities designed by the small groups for the purpose of soliciting feedback from the larger study group. 10. Create the small groups prior to the first live session, and provide an opportunity in every live session for them to work together in breakout rooms. Provide outcomes for each breakout session. 11. Participants received an extensive participant guide plus supplemental materials as necessary. Multiple documents created confusion, and in future deliveries the researcher will combine them into one document. Additionally, an example of a collaborative work plan for the small groups PLUS an example of what they are expected to produce will be included with the materials. (The participant guide is included in Appendix IV.) Summary The study attempted to provide an immersive experience in order to create an attitudinal change (belief that higher order thinking can be achieved using the virtual classroom) and a behavioral change (the ability to design and deliver breakout room activities for the virtual classroom) among study participants. The research study, while successful, uncovered many issues involved with the successful adoption of virtual classrooms for high-order learning, including the need for getting to know you activities, strong activity instructions, and well-organized documentation. The adoption of this training technology will only continue to grow, and with it the need for best practices and guidelines for accomplishing the desired instructional results. The researcher anticipates that organizations will need to adopt a pragmatic change
management approach in order to convince training teams and potential students that the quality of the learning experience in the virtual classroom can meet or exceed the quality achieved in a traditional environment. The key to acceptance will be equipping course designers with the training, time, and technologies they need to design interactions that deliver more than knowledge objectives and then using these well-designed interactions to convince students that they can be successful.