A Didactics-Aware Approach to Management of Learning Scenarios in E-Learning Systems

Transcription

1 A Didactics-Aware Approach to Management of Learning Scenarios in E-Learning Systems Dr. Denis Helic

2 To A. A heart whose love is innocent! - Lord Byron

3 A Didactics-Aware Approach to Management of Learning Scenarios in E-Learning Systems Postdoctoral Lecture Thesis submitted by Dr. Denis Helic Institute for Information Systems and Computer Media (IICM) University of Technology Graz Austria November 2006 Copyright 2006 by Denis Helic The work presented in this thesis has been carried out from 2001 until 2006 at Institute for Information Systems and Computer Media, University of Technology Graz.

4 Ein didaktisch-orientierter Ansatz zur Verwaltung von Lernszenarien in E-Learning Systemen Habilitationsschrift vorgelegt von Dr. Denis Helic Institut für Informationssysteme und Computer Media (IICM) Technische Universität Graz Österreich November 2006 Copyright 2006, Denis Helic Diese Habilitationsschrift ist in englischer Sprache verfasst. Die Arbeit zu dieser Habilitation wurde von 2001 bis 2006 am Institut für Informationssysteme und Computer Media der Technischen Universität Graz durchgeführt.

5 Abstract The first generation E-Learning systems were strongly technology-centric. As such, these systems typically neglected the didactic component of E-Learning. Not surprisingly, the results of the first studies that investigated improvements in efficiency of knowledge transfer, learning outcome, or users satisfaction in such E-Learning systems were very disappointing. These studies clearly showed that didactics in E-Learning should play a crucial role, and promising didactic approaches such as collaboration, project-oriented learning, or experientiallearning need to be addressed by the next generation of E-Learning systems. In the first part of this paper an innovative system called WBT-Master is presented. WBT-Master applies technology only as a vehicle that supports didactic aspects of E-Learning in the form of so-called learning scenarios. These scenarios might be seen as a particular way of how different users work with the system s tools and educational material to achieve a particular learning goal. The evaluation of WBT-Master showed tremendous improvement in learning effectiveness when compared with the first generation E-Learning systems. The second part of the paper discusses difficulties experienced during development of WBT-Master. Mainly, these difficulties were related to tedious and error-prone process of software development of tools that supported different learning scenarios. The main reason for such difficulties was an informal model for representing learning scenarios. To improve this situation WBT-Master introduced a generic formal model for describing learning scenarios and generic tools for executing such scenarios. In the last part of the paper an approach for imposing a procedural structure on top of learning scenarios is presented. This approach reflects the process-oriented nature of learning in general and E-Learning in particular. Therefore, WBT- Master was extended by means of a so-called Learning Process Manager tool. This tool provides a fully-fledged technological support for management of collaborative learning processes and allows definition, execution, and analysis of such learning processes.

6 I hereby certify that the work presented in this paper is my own and that work performed by others is appropriately cited. Ich versichere hiermit, diese Arbeit selbstständig verfasst, andere als angegebenen Quellen und Hilfsmittel nicht benutzt und mich auch sonst keiner unerlaubten Hilfsmittel bedient zu haben.

7 Acknowledgments Everyone at the IICM has been ready to provide me with valuable help, comments and feedback. Special thanks are due Dr. Christian Gütl, Victor Manuel García-Barrios, and Felix Mödritscher for much fruitful discussion. Above all, I would like to thank Prof. Hermann Maurer and Prof. Nick Scherbakov for providing me with ideas and suggestions for the work presented in this paper. Denis Helic Graz, Austria, November 2006

8 Table of Contents 1 Introduction A Didactics-Aware Approach to E-Learning Introduction Current Trends in E-Learning WBT-Master: A Didactics-Aware Approach to E-Learning Web-Based Reading Web-Based Tutoring Knowledge Profiling Knowledge Mining Project-Oriented Learning Evaluation of WBT-Master Concepts CORONET Project Evaluation Approach Evaluation Results in Corporate Environment Evaluation Results in Academic Environment Conclusion and Future Work Learning Scenarios in E-Learning Introduction Management of Learning Scenarios Problems with Scenario Management: Case Study Problems with Scenario Management: Case Study Problems with Scenario Management: Conclusions Representation of Learning Scenarios Meta-model Domain Models Formal binding: XML User Interface Processing of Scenarios... 52

9 3.4 First Experiences with Formal Representations of Learning Scenarios Experiences from Ephras Experiences with WBT-Master Conclusions and Further Work Technology-Supported Management of Collaborative Learning Processes Introduction Technology-Aware Framework for Collaborative Learning Processes State-of-the-art in Management of Collaborative Learning Processes Supporting Learning Processes through Business Process Management Sample Learning Process in LPM Implementation Aspects of LPM Learning Process Execution Engine Worklist Manager E-Learning Services Learning Process Definition Module Conclusion An Alternative Approach to Learning Scenario Management: Applying Loosely Coupled Web Services Introduction Closed-World and Open-World Distributed Systems Web Services and Open-World Assumption Orchestrating REST Web Services Supporting Learning Processes with REST Web Services Providing an Integrated User Interface for REST-based LPM Conclusion Bibliography...103

10 List of Figures Figure 2.1: Annotations in WBT-Master Figure 2.2: Working with Learning Actions Figure 2.3: Accessing Documents Anchored to Background Knowledge Figure 2.4: Retrieving Documents Anchored to a Specific Category Figure 2.5: Project Management Tool in WBT-Master Figure 3.1: Drag-and-drop exercise to support phrase meaning identification Figure 3.2: Thematic upload tool for management of student examples Figure 3.3: Graphical representation of a part of a QTI domain model Figure 3.4: Wizard-like user interface of the QTI model Figure 3.5: Setting attributes of a multiple-choice question Figure 3.6: Relation between a domain model and a captured learning scenario Figure 3.7: Editing of an automatically created QTI quiz Figure 4.1: Generic Learning Processes Framework Figure 4.2: Learning Process for Project-Oriented Learning Figure 4.3: LPM Manager Architecture Figure 4.4: Student Participation in a Learning Process Figure 4.5: Teacher Participation in a Learning Process Figure 4.6: Analyzing a Learning Process Figure 4.7 Selecting a learning activity for a learning scenario Figure 4.8 Defining properties of a learning activity... 90

11 List of Tables Table 4.1: Dynamics of Collaborative Learning Processes... 65

12 List of Listings Listing 3.1: XML-based representation of multiple-choice questions Listing 4.1 WSDL of Worklist Manager Service Listing 4.2 Part of a BPEL process that adds a Testing activity for a student Listing 4.3 SOAP Message from BPEL Engine for Worklist Manager Service Listing 4.4 SOAP Message from Worklist Manager Service for BPEL Engine Listing 4.5 WSDL of Get Test Result WBT-Master Service Listing 4.6 SOAP message from BPEL Engine to a WBT-Master service Listing 4.7 SOAP message from a WBT-Master service to BPEL Engine Listing 4.8 Decision rule in a BPEL process Listing 4.9 Learning process defined with the pedagogical vocabulary Listing 5.1 Typical REST URIs Listing 5.2 XML format for describing students Listing 5.3 XML format for describing courses and referencing students Listing 5.4 Orchestrating REST Web Services Listing 5.5 Adding a Test work item using REST Web services Listing 5.6 Defining a rule in a learning process based on REST Web services... 99

13 1 Introduction This paper presents the research and development work carried out by the author in a number of research projects that dealt with E-Learning and E-Learning systems. These projects include CORONET (IST ) and Ephras ( CP SI-LINGUA-L2) that were funded by the European Commission and a number of internal projects conducted with students in several university courses at University of Technology Graz. Generally, the first generation E-Learning systems were so-called course management systems. These systems utilized a well-known online course model that provided learners with a Web-based access to courses available in the system. In addition, these first E-Learning systems provided tools for teachers to manage online courses. Typically, teachers uploaded learning material to the system and structured that material as a number of online courses. Such online courses were composed out of a number of Web pages interrelated by means of computernavigable links [Oliviera et al. 2001]. Thus, online course model did not differ at all from a standard Web model except for the fact that the content in the first generation E-Learning systems was educational content. The research and development efforts in that first phase of E-Learning concentrated on the technological aspects of the field. For example, the system development efforts were mainly concerned with providing tools to enable easy development, access, and work with online courses, i.e. Web-based educational content. Also, the first standardization efforts in the field concentrated on structuring, reuse, and interoperability issues in E-Learning at the level of educational content. As such all of these efforts completely neglected the didactical component of E-Learning, i.e. the focus was on the E part and not on the Learning part of E-Learning [Mioduser et al. 2000]. However, the results of the first studies that investigated improvements in efficiency of knowledge transfer, learning outcome, or users satisfaction in E- Learning were very disappointing. These studies showed that the main reason for - 1 -

14 Introduction such disappointing results was the primitive online course model that, from the didactical point of view, involved learners only in simple Web-based reading of the educational content. Moreover, the studies argued that didactics in E-Learning should play a crucial role, and promising didactic approaches such as collaboration, project-oriented learning, or experiential-learning need to be addressed by the next generation of E-Learning systems [Mioduser et al. 2000; Leasure et al. 2000; Oliver et al. 2002]. In the scope of the CORONET project a novel E-Learning system called WBT- Master has been implemented by the IICM. The system was strongly didactically oriented and supported a number of collaborative didactic approaches by means of innovative Web-based tools. Basically, each of such tools reflected a particular learning scenario in a Web-based environment. Thus, the system included scenarios such as Web-based tutoring, knowledge profiling, knowledge mining, collaborative writing, or project-oriented learning. Learning scenarios were based on an informal model that included different user roles such as learners and teachers, a particular sequence of learning activities performed by the learners, the educational content, and the system tools needed to carry out the learning activities. The implemented learning scenarios reflected the E-Learning requirements of the corporate and academic partners from the project. The project was concluded with an evaluation of WBT-Master and the developed didacticsaware approach to E-Learning. The obtained evaluation results clearly showed tremendous improvements in learning effectiveness, user satisfaction, or even in cost-benefit ratio of efforts invested in E-Learning. Although such results were very promising a number of technical problems with such a didactics-aware approach have been identified. Mainly, these problems were related to tedious, difficult and error-prone process of software development of tools that supported different learning scenarios. The main reason for such difficulties was the informal model for representing learning scenarios and informal nature of collecting of the requirements for such scenarios - typically achieved through informal user-developer dialogues. The result of such informal dialogues contained inconsistent and incomplete information because of misunderstandings and the complexity of the interactions within a scenario. Consequently, tools development was suboptimal and a number of iterations were required in order to achieve better results. To solve such problems WBT-Master was extended within the scope of the Ephras project and an internal university project called ivisice. The developed extension included a generic formal model for describing learning scenarios in WBT-Master. On top of that model a wizard tool was implemented to support collecting of the user requirements. Additionally, a generic configuration tool was implemented. This tool mapped the collected user requirements (automatically or semi-automatically) onto the system tools and provided an integrated user - 2 -

15 Introduction interface to support a particular learning scenario. Again, the first evaluation results of the developed model and tools in these two projects were very positive. Nevertheless, the experiences from these two projects showed that such a learning scenario model had one didactics-related deficiency. Basically, the model did not take into the account the dynamics of a learning scenario. Thus, teachers could select the didactical features which they would like to include in a learning scenario but they could not impose a procedural structure on top of these features. For example, collaborative writing learning scenario includes learning activities such as writing, reading, reflecting, commenting, and discussion that are organized in a certain execution procedure, i.e. firstly, learners need to read a particular document; secondly, learners write their own documents; thirdly, the teacher provides comments; lastly, learners are involved in reflecting activity. Thus, there exists a process-oriented execution sequence within this particular learning scenario. Obviously, such execution sequences should be also supported by an E-Learning system. In order to support such a process-oriented nature of learning scenarios in E- Learning environment WBT-Master was extended by means of a so-called Learning Process Manager tool. This tool provides a fully-fledged technological support for management of collaborative learning processes and allows definition, execution, and analysis of such learning processes. The tool is based on technologies such as workflows, business processes, and Web-based technologies for distributed software development, most notably Web services. The rest of this paper is organized as follows. Chapter 2 presents the initial work on WBT-Master to support didactics-aware learning scenarios. That work was carried out in the scope of the CORONET project. Chapter 3 presents the work on formalizing and automatically managing learning scenarios to simplify the work needed for development of these scenarios in an E- Learning system. This work was carried out in the scope of the Ephras and ivisice projects. Chapter 4 presents the extension of the learning scenario model to reflect the process-oriented nature of learning scenarios. Technically, this extension is based on a so-called Business Process Management technology and Web services technology. Chapter 5 discusses some of the technical difficulties related to the application of the Business Process Management technology and related technologies such as Web services. This chapter proposes an alternative technological solution for management of collaborative learning processes based on a novel architectural style for distributed software development in a Web-based environment. That - 3 -

16 Introduction style includes a so-called Representational State Transfer (REST) model for developing Web services and Asynchronous JavaScript and XML (Ajax) for developing rich-client Web applications. Lastly, Chapter 6 provides some conclusions and suggestion for the future work in this field

17 2 A Didactics-Aware Approach to E-Learning This chapter is a slightly modified version of an article that has been published as a book chapter by Springer-Verlag in Studies in Fuzziness and Soft Computing edited by Claude Ghaoui, Mitu Jain, Vivek Bannore [Helic et al. 2005a]. This chapter argues that the current E-Learning systems, as well as the recent standardization efforts in this field are strongly technology-centric. Because of this, such endeavors usually neglect the didactic component of E-Learning and are therefore completely didactic-neutral. However, recent research in the field shows that in order to achieve improvements in efficient knowledge transfer, learning outcome, or users satisfaction didactics plays a crucial role. For example, such promising didactic approaches as collaboration, project-oriented learning, or experiential-learning need to be addressed by E-Learning systems and standards. In this chapter an innovative system called WBT-Master that was built at Institute for Information Systems and Computer Media (IICM) at Graz University of Technology is presented. In WBT-Master technology is applied only as a vehicle that supports didactic aspects of E-Learning. The main idea of WBT-Master is in the use of both conventional and innovative tools compatible with the current Web and E-Learning standards to support didactics in a Web-based environment and in that way facilitate, more efficiently, transfer of knowledge from people who posses that knowledge to people who need to acquire it. 2.1 Introduction The experience collected in a number of E-Learning projects conducted at IICM shows that most of the system development and standardization efforts in E- Learning field concentrate on the technological aspects of the field. Usually, these - 5 -

18 A Didactics-Aware Approach to E-Learning efforts neglect to a great extent the didactic side of E-Learning. For example, the major commercial E-Learning systems, such as WebCT [WebCT 2004] or Blackboard [BB Blackboard 2004] are mainly concerned with providing the tools to enable easy development, access, and work with Web-based educational material. From the didactic point of view such systems are based on learners taking a part in what can be simply called Web-based reading of educational material. Similarly, the standardization efforts such as Sharable Content Object Reference Model (SCORM) [SCORM 2004] developed by the international standardization bodies deal mainly with interoperability issues in E-Learning at the content level. Thus, such standards focus on creation, distribution, exchange and reuse of educational material in different E-Learning systems. However, SCORM compliant educational material usually does not carry any additional didacticrelated information. Rather it only defines a simple navigational structure which prescribes how to navigate through the content. Such a navigational structure supports yet again only the simplest Web-based consumption of the educational content - Web-based reading. However, this chapter argues E-Learning should support a wide range of different didactic approaches to enable efficient knowledge transfer in a Web-based environment. Here Web-based reading might be seen as just one (and for that matter the simplest one) of such didactic approaches. In such an approach to E- Learning the Web-based technology is applied only as a vehicle that supports didactic aspects of E-Learning, and ceases being the central part of it. Thus, the key here is in the use of both conventional and innovative tools compatible with the current Web and E-Learning standards to incorporate didactics into E- Learning and in that way facilitate, more efficiently, transfer of knowledge from people who posses that knowledge to people who need to acquire it. Technically speaking, the simplest way of introducing didactic aspects to E- Learning is by implementing tool support for a number of teaching and learning scenarios into E-Learning systems. For example, such typical scenarios as projectbased learning, goal-oriented learning, or experiential-learning should be supported by tools and systems to cover a wide range of day-to-day training situations in both academic and corporate environment. This chapter describes the efforts to implement this support into the WBT-Master, an innovative E-Learning system that was developed at IICM. The system supports a number of typical, as well as innovative teaching and learning scenarios applicable in E-Learning. The remainder of this chapter is organized as follows. The next section describes in more details the current trends in E-Learning system development, research, - 6 -

19 A Didactics-Aware Approach to E-Learning and standards clearly identifying the above mentioned technology-centric problem of E-Learning. The third section provides an introduction to WBT-Master and lists teaching and learning scenarios supported by the system. The consequent sections describe in more details the listed scenarios at both levels - the didactic and the technological level. Finally, the chapter presents the results of an evaluation of the WBT-Master system and its concepts. The evaluation was conducted with hundreds of users in both academic and corporate environment and concentrated on estimating the improvements in learning outcome by using the traditional classroom approaches, the current E-Learning approaches, and the innovative approaches presented in the chapter. Moreover, issues such as learning experience, user satisfaction, usability, and others were addressed by the evaluation. 2.2 Current Trends in E-Learning Recent surveys on the number of installations and distribution of E-Learning systems at universities in Europe [Paulsen et al. 2002], Australia [Byrnes and Ellis 2004] and the USA [CIC 2002] show that the two most popular and used systems are WebCT and Blackboard. Approximately, the installations of these two systems constitute more than 60% of all WBE system installations. Both of these systems can be categorized as so-called course management systems, i.e., they utilize the well-known online course model. Usually, all the tools in such systems are optimized to support easy creation, publishing, update, and access to Web-based courses. Such Web-based courses are composed of a number of Web pages interrelated by means of computer-navigable links. Thus, the authoring tools allow authors to upload Web pages created on their local sites, or to create new Web pages directly with the system via integrated Web page editors. At the next step, the pages are related to each other and the publishing procedure makes the finished course accessible for learners. Additionally, the published courses are associated with a discussion forum which provides the communicational context for the learning sessions [Oliviera et al. 2001]. From the didactic point of view the facilities offered by these systems leave much to be desired. Usually, these systems support only traditional forms of teaching (e.g., a teacher prepares educational material that learners need to read) and encourage poor instructional design models. Consequently, the learning outcome and the users satisfaction (of both teachers and learners) are quite low [Oliver et al. 2002]. For example, a study conducted at the College of Nursing, University of Oklahoma, compared the learning outcome in a traditional vs. Web-based baccalaureate nursing research course. The study showed the following disappointing results. Overall, there was no significant difference in examination scores between the two groups as far as the three multiple-choice examinations and the course grades are concerned. Students who reported that they were selfdirected and had the ability to maintain their own pace and avoid procrastination - 7 -

20 A Didactics-Aware Approach to E-Learning were most suited to Web-based courses [Leasure et al. 2000]. Clearly, these results need to be improved tremendously. The main reason for this situation lies in the fact that such systems are strongly technology-centric. Basically, the current E-Learning systems try to solve technology-based problems of E-Learning, such as authoring of educational material, reuse of Web-based courses, user management, and efficient learners progress tracking, and so on. Thus, the role of didactics is strongly neglected in such systems. For example, Mioduser refers to this situation as one step ahead for the technology, two steps back for the pedagogy. However, the key must be in research and development of novel Web-based educational models and in the support of the current pedagogical approaches (e.g., use of inquiry-based activities, application of constructivist learning principles, and use of alternative evaluation methods) [Mioduser et al. 2000]. Similarly, other research work suggests development of pedagogy-aware Webcourses emphasizing that the strategies that enhance learning in the traditional classroom should be replicated in Web-based learning sessions. For example, these strategies include but are not limited to accommodating diverse learning styles, incorporating a good study guide with a content section, providing a communicative network and establishing a review process [Hobbs 2002]. Clearly, the current E-Learning systems are far away from supporting such methodologies. On the other hand, organizations such as IEEE Learning Technology Standards Committee, IMS Global Learning Consortium, and Advanced Distributed Learning Initiative (ADL) are working on standardization efforts in E-Learning. These efforts include the following standards: Learning Object Metadata (LOM) standard [LOM 2002] for describing educational material with standardized metadata. This metadata should support learners and teachers in retrieving relevant educational material in an easy and efficient way. Further, it should provide the technical background for interoperability of educational material from different E- Learning systems. Thus, reuse and exchange of educational material between two different E-Learning systems is encouraged by this standard. SCORM suite of standards, such as SCORM packaging or SCORM Simple Sequencing [SCORM 2004a]. These standards address issues of packaging Web-courses so that interoperability, reuse, and exchange of Web-based courses between different E-Learning systems are guaranteed. For example, SCORM packaging provides a standardized way of organizing the educational content into items, small packages, or even complex structures, and prescribes how such content can be navigated. SCORM Simple Sequencing goes one step further by specifying so-called learning paths, which can branch according to the current learning situation

21 A Didactics-Aware Approach to E-Learning IMS standards [IMS 2004], such as IMS Metadata, IMS Content Packaging or IMS Simple Sequencing, which address similar issues as LOM or SCORM standards. Actually, many of the IMS standards, such as IMS Simple Sequencing are included in other standards such as SCORM Simple Sequencing. Although such standardization efforts have many advantages, such as sharing and reuse of educational material, standardized way of packaging of educational material, flexibility in content presentation, interoperability across systems, they also have some disadvantages. The basic disadvantage is the total lack of addressing didactics and pedagogy in E-Learning. For example, SCORM claims to be pedagogically neutral, which means that it is impossible to create SCORM packages that relate to some didactic approach, say project-oriented, or problemsolving learning approach [Jonassen and Churchill 2004]. In other words SCORM packages can not capture didactic relations between their components. However, as recent research studies show incorporating didactics into E-Learning leads usually to far better results in learning outcome, learners and teachers satisfaction, learners community building, and so on. For example, Hirumi shows in his study [Hirumi 2002] that careful planning and design of interactions in E- Learning, where learners are supervised by means of immediate feedback, discussions and clear didactic and learning goals leads to improvement in learning outcome. Similarly, the study with deployment of project-oriented collaborative didactic approach conducted by King [King and Puntambekar 2003] shows tremendous improvements in building a community of learners, which helped to solve problems related to the project at hand. In the future, one of the keys to the learning process in E-Learning will be communication between learners themselves, learners and teachers, and the formation of learning communities held together by a common learning goal, which is modeled by a sound didactic approach. 2.3 WBT-Master: A Didactics-Aware Approach to E-Learning In WBT-Master didactic approaches are referred to as teaching or learning scenarios. These scenarios might be seen as a particular way of how different types (roles) of users work with the system, the system s tools and educational material available in the system to achieve a particular learning goal. Thus, each scenario can be described by: A particular way (i.e., a story) of how to achieve the learning goal, The user roles that are involved in the story, The system tools needed to support the story, Educational material that is needed to achieve the learning goal

22 A Didactics-Aware Approach to E-Learning For instance, the above mentioned Web-based reading scenario can be defined as the following teaching scenario: An author has a group of learners that need to improve their knowledge on a certain subject. Thus, the author prepares a number of Web pages containing relevant educational content and connects these pages with links in a navigable structure (i.e., course). After the course has been created the author publishes it in the system. Now, the learners access the published material and read through it to improve their knowledge about the subject. During their work with the published material the learners communicate with the author via the attached discussion forum. Additionally, the author tracks the progress of the learners by means of different statistic tools. The user roles involved in the scenario are authors and learners. The system tools needed to support the scenario are the authoring tool for preparing Web-based educational material (i.e., courses), and a standard Web browser to access and work with educational material. Educational material comes in the form of a number of Web pages, which contain relevant educational content. The pages are connected by links into a navigable structure. The teaching and learning scenarios, which are implemented in WBT-Master try to take into account recent advancements in the traditional classroom didactics, as well as in the technology enhanced didactics. These scenarios incorporate such promising didactic approaches as project-based learning, problem-solving, constructivist approaches, collaboration, and so on. This chapter presents the following teaching and learning scenarios from WBT- Master: Web-based reading - this basic E-Learning scenario was extended in WBT-Master by sophisticated communicational and collaborative features such as annotations. Web-based tutoring - a teaching scenario where a tutor works with a group of learners in both synchronous and asynchronous mode, leading them to achieve a particular learning goal. Knowledge profiling - a scenario supporting the acquisition, structuring, and reuse of extracted expert knowledge. Knowledge mining - a learning scenario where learners are supported in exploring extracted expert knowledge by means of personalized knowledge retrieval facilities. Project-oriented learning - a learning scenario where a group of learners works together on a project, e.g., a software engineering project. In the remainder of the chapter each scenario is described according to the above introduced template, i.e., the user roles, the system tools, educational material and

23 A Didactics-Aware Approach to E-Learning the story of the scenario are given. After the scenario was defined a discussion of didactic aspects of that approach is presented. Then, the technical issues such as software requirements, technical problems and obstacles, as well as possible solutions are discussed. Finally, the chapter presents an example and a screenshot of a typical learning session with that particular scenario. 2.4 Web-Based Reading The Web-based reading scenario was already defined in the previous section. The support for this simple teaching scenario in WBT-Master closely reflects the similar support in other E-Learning systems, such as WebCT or Blackboard. However, there are also a few significant differences. Didactically, the Web-based reading scenario in WBT-Master was extended by the promising collaborative facilities, such as annotations and synchronous and asynchronous communication [Mason et al. 1999]. Thus, the learners and the author can add and change the content by annotating it for themselves or others. Other users can even annotate the notes previously made, in this way activating a powerful communication channel. Each annotation has a certain type, such as Comment, Question, Answer, etc., which provides a communicational context that can be very important in the learning process [Sorensen and Takle 2001]. Moreover, annotations can also take the form of links, i.e., material can be linked together by the learners for their own benefit and for the benefit of the whole group. Thus, learners themselves contribute to the content on-the-fly (see Figure 2.1). Further, the Web-based reading scenario in WBT-Master supports other ways of synchronous and asynchronous communication between the learners and the author. These communicational facilities include chat rooms and the attached discussion forum. Note, that all the annotations that were made previously within the context of the educational content are also accessible via the discussion forum. Technically, educational material in the Web-based reading scenario is SCORM compliant. This ensures modularity, reuse, and interoperability of the educational content units. For example, suppose that we developed an educational unit about Relational Data Model. Now, this unit might be reused in a number of different contexts. For instance, it can be reused in the context of another educational unit dealing with Databases in the practical sense, but it can also be reused in the educational unit dealing with the theoretical aspects of Data Models

24 A Didactics-Aware Approach to E-Learning Figure 2.1: Annotations in WBT-Master 2.5 Web-Based Tutoring The Web-based tutoring scenario might be defined as the following learning scenario: A tutor has a group of learners that need to achieve a particular learning goal, e.g., the learners need to acquire some knowledge in a particular subject matter or they need to learn how to solve a particular kind of problems. The tutor defines a learning path, which the learners need to follow in order to achieve the learning goal. Such a learning path is a sequence of learning actions that need to be accomplished in a step-bystep manner by the learners. Each learning action comes with some educational material which should be consumed by the learners at that particular step. Alternatively, a learning action might be associated with a test that the learners need to pass, a request for publishing a document, a request to solve a particular problem, or simply a request to communicate with the tutor. Thus, the learners access the learning path and work through the learning actions consuming learning resources, working on tests, publishing documents, and so on. During this time the tutor can provide feedback to the learners by evaluating tests, answering their questions, and similar. Finally, the tutor may alter the learning path (e.g., insert a new learning action) as long as the learning situation requires it. Additionally, the learners and the tutor can communicate via the attached discussion forum. The user roles involved in the scenario are tutor and learners

25 A Didactics-Aware Approach to E-Learning The system tools include the authoring tool for developing learning paths as sequences of learning actions. Additionally, the tool for managing a library of educational material needed for a particular learning session is at the tutor s disposal. On the other hand, the learners need only a standard Web browser to access and work with educational material, to make tests, or to publish their documents. Educational material can be of any kind, i.e., courses, documents, discussion forums available in the system or external Web resources such as external Web pages. The Web-based tutoring scenario reflects the well-known goal-oriented didactic approach [Uljens 1997]. Thus, the tutor leads the learners to achieve a particular learning goal. For example, a particular learning goal for software engineering students might be to learn how to write the user requirements document for a software system. In the Web-based tutoring scenario the learning goal is achieved by following a predefined sequence of learning actions, i.e. the learning path. Since there are different types of learning actions, such as reading, writing, solving a problem, answering questions and others, the learning process can be based on sophisticated instructional models (see Figure 2.2). For example, the above mentioned software engineering students, after having read more theoretical documents, might become involved in writing a sample user requirements document to gain practical experience. Note here the difference between the Web-based tutoring and the Web-based reading scenario. In the Webbased reading scenario the learners are supposed to reach their learning goal by simply following the navigational sequence and reading the educational content. Consequently, the Web-based reading scenario cannot prescribe a writing assignment for the software engineering students as the Web-based tutoring scenario can. Another important aspect of the Web-based tutoring scenario is the immediate feedback by the tutor to the current learning situation. At each particular step of the learning session the tutor can provide feedback to the learners by communicating with them, evaluating their contributions, or alternating the learning path if new learning actions need to be inserted. For example, the tutor recognizes that the learners did not understand some concept entirely (e.g., by looking at the test result) and that they need some additional information. Thus, the tutor decides to insert a new learning action into the learning path attaching to it a document that provides the needed information. Note that the feedback can be provided for the whole group of learners, as well as for a single learner. In this way, the learning actions are customized to the current learning needs, learning situation, knowledge level, and learning preferences not only of the whole group

26 A Didactics-Aware Approach to E-Learning but also of a particular learner. Thus, each learner s learning experience can be highly personalized by the tutor. Figure 2.2: Working with Learning Actions From the technical point of view, the tutor is involved in managing the library of educational material and in creating and manipulating the learning path. Note that the tutor is supposed to reuse all educational material available in the system by including it into the library. This material is then being referenced from within the learning path. To ensure interoperability between different scenarios and to enable reuse we decided again to apply the SCORM standard for defining the educational content (i.e., packaging of the content) and the learning path. The SCORM standard includes the simple sequencing model, which provides means for defining learning paths and rules for choosing between different alternatives. Although the simple sequencing model cannot capture all aspects of the Webbased tutoring scenario (e.g., altering of the learning path based on communication between the tutor and the learners) it can be seen as a solid basis for further development. Note also that this solution works as an authoring tool for such simple sequence models because the sequences are defined by the tutor onthe-fly taking into account the current learning situation. This can be seen as the additional value of the tool because authoring of such sequences before any learning session starts is usually a very hard task

27 A Didactics-Aware Approach to E-Learning 2.6 Knowledge Profiling The knowledge profiling scenario is defined as the following learning scenario: A teacher (e.g., an author or a tutor) has a group of learners that need to improve their knowledge on a certain subject. The organization with which the teacher and the learners are affiliated manages a large repository of extracted expert knowledge in the form of documents, external Web pages, lessons learned, discussion forums, etc. To ensure that the learners learn from the expert knowledge the author develops domain ontology of the subject and classifies the expert knowledge according to the ontology. Finally, the learners can access and work with the classified knowledge. The user roles involved in the scenario are teachers (either authors or tutors, or both), learners, and indirectly experts. The system tools which are needed include the authoring tool for developing domain ontology, the authoring tool for classifying extracted expert knowledge according to the developed domain ontology, and a standard Web browser for learners to access and work with the classified expert knowledge. Educational material in this scenario is extracted expert knowledge coming in the form of internal documents, external Web pages, discussion forums, etc. From the didactic point of view, this scenario addresses a few issues of the socalled experiential-learning. The experiential-learning is related to learning with experiences [Dewey 1938; Dirkx and Lavin 1991]. The main concern of the experiential learning is how to transfer these experiences efficiently. Such experiences are mostly provided by experts, where the expert experience results from many years of practice. The problems in the experiential learning come from the fact that the knowledge of experts is somehow routine and difficult to make explicit or understandable to novices. Some of the reasons for this situation are the lack of the background knowledge of the domain (the declarative knowledge) and the lack of anchoring between experiences and the declarative knowledge [McKay et al. 2002]. The knowledge profiling scenario tries to bridge this gap in the experiential learning by providing a high-level declarative description of the domain (i.e., the domain ontology), and by linking the expert experiences to the ontology, i.e., by classifying the experiences to the categories and the relations of the domain ontology. For example, suppose we have the extracted expert knowledge in the domain of databases and information systems. This knowledge is comprised of, let say, a single general document about information systems and a single lessons learned document on the practical development of database systems. The domain ontology might include two categories: the Information Systems and the Databases category. Since database systems are a special kind of information

28 A Didactics-Aware Approach to E-Learning systems the domain ontology might relate the Information Systems category to the Databases category by means of the includes relation. Additionally, the ontology might include the inverse relation of the includes relation, i.e., the isa-kind-of relation. Finally, we might classify the general document about information systems to the category Information Systems and the lessons learned document to the Databases category. Note, that the two documents are now automatically anchored within the background knowledge, i.e., they are explicitly related by means of the includes and its inverse is-a-kind-of relation (see Figure 2.3). Technically, the implementation of the knowledge profiling scenario needs to meet the following requirements. Firstly, the system must support the development of domain ontology or seamless integration of domain ontology developed with external ontology editors. To ensure interoperability between the system and external tools domain ontology should be developed by means of standardized knowledge representation techniques, such as recently developed RDF Schema [RDF 2004] and OWL [OWL 2004] languages. Secondly, the system needs to support the teacher during the classification of the extracted expert knowledge by means of automatic and semi-automatic methods. For example, the system might suggest to the teacher that a particular document should be included into a specific domain category. There are a few different ways to support automatic or semi-automatic document classification, such as metadata management or full-text processing. WBT-Master applies metadata for this purpose, since implementing document classification is usually very hard in an E-Learning environment. Usually, such an environment deals with heterogeneous documents (e.g., Web pages, discussion forums, internal documents in different formats, etc.), which makes supporting of full-text processing very difficult. Also, metadata management and especially metadata gathering in such an environment can be very expensive since the users of the system need to provide metadata manually. WBT-Master applies a semiautomatic approach for metadata gathering. Thus, the system manages sophisticated user profiles, which contain information of users field of expertise, users general interests, users current involvement in the learning and teaching processes, and so on. Then, the system tries to apply this information to automatic generation of metadata. For example, suppose we have an expert in Databases. The expert declares the Database expertise in his/her user profile. Now, whenever this expert contributes a document to the system, the system automatically adds a metadata description to the document stating explicitly that this document deals with Databases. Then, this information can be used during the classification process

29 A Didactics-Aware Approach to E-Learning Figure 2.3: Accessing Documents Anchored to Background Knowledge Lastly, the system needs to support learners in their work with the domain ontology and the classified expert knowledge, i.e., the categories of the domain should be searchable and navigable. For example, the learners might want to search for all documents belonging to the Databases category, or they can navigate through the domain ontology and in that way reach the Databases category and its documents (e.g., by following the link includes emanating from the Information Systems category). 2.7 Knowledge Mining The knowledge mining scenario might be seen as a refinement of the knowledge profiling scenario. Thus, this scenario is built up on the same infrastructure, i.e., the domain ontology and the classification of the extracted expert knowledge by means of this ontology. The main difference is in the way how learners access that knowledge. In the knowledge profiling scenario learners had to navigate or search through the ontology categories to find the relevant information. Taking into account that typical domain ontology can include hundreds, even thousands, of categories and relations this might be seen as a rather tedious task. Moreover, in a typical training situation in a corporate environment learners need to find relevant information easily and quickly, and usually do not have time to navigate through thousands of categories. In such a common on-demand training situation, the knowledge mining scenario tries to provide support for learners in their initial access to relevant information

30 A Didactics-Aware Approach to E-Learning Didactically, the knowledge mining scenario addresses yet another issue related to the experiential-learning. This issue deals with the way how learners access relevant information, i.e., how they find such information in an efficient manner. From the technical point of view, the knowledge mining scenario extends the knowledge profiling scenario in the following way. In the knowledge profiling scenario we were mainly concerned with creation of the domain ontology and the classification of the expert knowledge to the categories from the ontology. One of the methods applied to automatic or semi-automatic execution of this process was a management of user profiles, and description of users field of expertise with metadata. In the knowledge mining scenario we extend the notion of user profiles by describing what are the fields of interest of each particular learner. This information is then used to facilitate the initial access to relevant information. For example, suppose a learner is interested in information systems. Now, whenever the learner accesses the categorized expert knowledge, say by navigation, the system provides links to all documents containing expert knowledge on the topic of information systems. Similarly, search mechanism profits from the same information by ranking documents dealing with information systems at the top of search results. Lastly, the system makes use of another important property of domain ontology, i.e., inference. Inference is a technique that supports deduction of new facts, e.g., automatic classification by investigating categories, relations and their properties in domain ontology. Recollect the example that we introduced above, i.e., we have two categories: the Information Systems category and the Databases category. These two categories are related by means of the is-a-kind-of relation, i.e., the Databases category is-a-kind-of the Information Systems category. Usually, the is-a-kind-of relation is defined as transitive, i.e., if A is-a-kindof B, and B is-a-kind-of C, then A is-a-kind-of C, thus allowing the principle of subsumption to be applied. Obviously, if a document is classified to the Database category then it can be (because of transitivity of the is-a-kindof relation) automatically classified to the Information Systems category. Now, whenever the learner who is interested in information systems can access that category and the system automatically provides the learner not only with links to the documents in information systems, but also with links to the documents in databases (see Figure 2.4)

31 A Didactics-Aware Approach to E-Learning Figure 2.4: Retrieving Documents Anchored to a Specific Category 2.8 Project-Oriented Learning The project-oriented learning scenario can be defined as the following learning scenario: A teacher (an expert, an author or a tutor) has a group of learners that need to gain a practical experience in project-based collaborative work, e.g., working in a group on a software project. The teacher initiates a projectbased learning session by creating a detailed project plan with the project steps and the time plan. At the next step, the teacher provides a sample project, which shows all the steps of a successfully executed project. Finally, the teacher provides a number of project alternatives for the learners. The learners constitute a number of teams, where each team selects one of the possible project alternatives as their practical example. The system provides communication tools, such as a discussion forum, a chat room, as well as collaborative facilities, such as version control system, annotations, tools to write project documents in collaboration, and so on. The teacher monitors the progress of the learners and provides feedback when necessary. The user roles involved in the scenario are teachers and learners. The required system tool is the integrated project management room, which provides facilities for creating and managing project plans, sample projects, project alternatives, as well as communicational and collaborative facilities

32 A Didactics-Aware Approach to E-Learning Educational material in this scenario consists of the project plan, the sample project, as well as external resources that may be linked to the project room via the discussion forum or annotation facilities. Didactically, project-based learning is a model of learning that organizes learning around projects. Usually, projects are complex tasks, based on challenging questions or problems that involve learners in design, problem-solving, decision making, or investigative activities, furthermore they give learners opportunity to work relatively autonomously over extended periods of time; and culminate in realistic products or presentations [Jones et al. 1997]. Other defining features found of project-based learning paradigm include authentic content, authentic assessment, teacher facilitation but not direction, explicit educational goals, cooperative learning, reflection, and incorporation of adult skills. Crucial for a successful and effective application of such a project-based learning paradigm is the careful developing and planning of effective projects. The basic properties of such effective projects might be summarized as follows [Thomas et al. 1999]: Learners should be put at the center of the learning process. The project work is central to the curriculum. The project must motivate learners to explore important topics on their own. Project management should be accomplished by using appropriate tools, such as computer-based project management tools. The project outcome or the result that learners need to produce must include learning techniques such as problem solving, in-depth investigation of topics, research, reasoning, and so on. The project should include outcome alternatives that learners might choose from, or that they can work on one after another applying the experience they gained before. The project must be collaborative, that is learners might work together in small groups, or they can present and discuss their partial and complete results with other learners at any time. Let us look now at an example of a project-based learning course. In a study reported by Barron [Barron et al. 1998], learners worked for five weeks on a combination of problem-solving and project-based learning activities focused on teaching learners how basic principles of geometry relate to architecture and design. The problem-solving component involved helping to design a playground in a simulated computer aided environment. The project-based component involved designing a playhouse that would be built for a local community center. Following experience with the simulated problem, learners were asked to create two- and three-dimensional representations of a playhouse of their own design and then to explain its features in a public presentation to an audience of experts

33 A Didactics-Aware Approach to E-Learning Recently, numerous research papers on project-based learning have been published showing the benefits of this learning paradigm for learners and teachers as well. For example, these reports show tremendous gains in learner achievements, large gains in learners problem solving capabilities, gains in learners understanding of the subject matter, perceived changes in group problem solving, work habits, and other project-based learning process behaviors [Thomas 2000]. Technically, the project-based learning must be supported by means of an integrated project management tool (see Figure 2.5). Figure 2.5: Project Management Tool in WBT-Master An implementation of such a project-management tool is a standard part of WBT- Master. This tool consists of the following components: A special document (curriculum) describing in few words the course and project motivation, problems that need to be solved, goals, etc. A special discussion folder providing a sample project with the definition of project plan, i.e., number of project steps and the time table for these steps. Each step is documented with a number of publications. A number of project discussion folders, which provide project alternatives for learners to chose from. These folders hold also all learner contributions. A number of collaboration and communication tools, such as online presence lists, chat rooms, discussion forums, etc. Evaluation tool for teachers evaluating learners work

34 A Didactics-Aware Approach to E-Learning 2.9 Evaluation of WBT-Master Concepts WBT-Master was developed within the scope of CORONET (Corporate Software Engineering Knowledge Networks for Improved Training of the Work Force) project funded by the European Union. The CORONET project was running from Mai 2000 until Mai The project consortium consisted of: Center for Advanced Empirical Software Research, the University of New South Wales, Sydney, Australia Atlante, Madrid, Spain DaimlerChrysler, Ulm, Germany Fraunhofer Institute for Experimental Software Engineering (IESE), Kaiserslautern, Germany Fraunhofer Institute for Computer Graphics (IGD), Darmstadt, Germany Highware, Paris, France Institute for Information Systems and Computer Media (IICM), Graz University of Technology, Austria Centro de Computacao Grafica, Coimbra, Portugal WBT-Master was mainly developed by the IICM. The application partners in the project were DaimlerChrysler, both Fraunhofer institutes, and Highware. These institutions deployed WBT-Master and evaluated it in a wide range of possible applications. Additionally, we at the IICM used the system for hundreds of university students during lectures at our university CORONET Project Evaluation Approach The CORONET project evaluation activities were performed through the following 3 phases [Trapp 2002]: Phase 1 (June - August 2001): In-depth assessment of the first WBT- Master prototype. The results from this evaluation were the main input for the enhancement of the CORONET methodology and infrastructure during the 2nd cycle of the CORONET project. Phase 2 (September April 2002): Continuous evaluation studies performed with the WBT-Master prototype in parallel with incremental enhancements of the product. Phase 3 (February - April 2002): In-depth assessment of the improved WBT-Master prototype. The three phases were performed in a systematic way during the project according to a detailed evaluation plan developed during cycle 1 of the project. The evaluation approach was finely tuned with the contribution of software and learning evaluation experts involved as members of the Pedagogic Advisory Board. The evaluation activities during phase 2 and 3 were monitored by additional requests derived from phase 1 results and from the comments of the

35 A Didactics-Aware Approach to E-Learning 2nd and 3rd CEC in-depth project reviews. General goals for WBT-Master evaluation were: Analysis of learning effectiveness: Evaluating the effectiveness of WBT- Master system in supporting knowledge sharing and collaborative learning. Usability analysis: Evaluating the perceived ease of use and the perceived usefulness of WBT-Master system. Cost-Benefit Analysis (CBA): Evaluating the cost-benefit ratio of using WBT-Master system. In order to evaluate the WBT-Master, the partners that conducted the evaluation chose a mixed evaluation approach, i.e., each partner did not focus on all of the goals, but selected one or more focus areas to which individually tailored evaluation processes were applied. To analyze learning effectiveness DaimlerChrysler, consistent with its role as a software development organization, focused on evaluating the effectiveness of WBT-Master in supporting continuing, self-directed, collaborative learning. The evaluation process was based upon the cognitive load theory and relied on a series of specifically designed evaluation sessions that were conducted in a specifically established evaluation laboratory setting, involving members of the research group as well as members of a business unit. Fraunhofer, consistent with its role as a research institute, focused on evaluating the effectiveness of WBT-Master in supporting collaboration and knowledge sharing. This was done by conducting two quasi-experiments that compared the efficiency and effectiveness of conducting similar tasks with and without using WBT-Master. Highware, consistent with its role as a training service provider, focused on evaluating the effectiveness of WBT-Master in supporting web-based learning by training and web-based experience sharing. Evaluation data was collected with the help of specifically designed questionnaires. To perform usability analysis Perceived Ease of Use (PEU) and Perceived Usefulness (PU) of WBT-Master from the point of view of end users was estimated. In order to analyze the PEU and PU end users were requested to answer related sets of questions. For data collection, specifically designed questionnaires were distributed to end users of WBT-Master at the end of the trial period for a particular learning scenario. In order to test the reliability of their analysis results Fraunhofer IESE calculated and interpreted Cohen s Kappa coefficient. DaimlerChrysler used the questionnaire ISONORM 9241/10, which was evaluated with respect to validity and reliability. The ISONORM questionnaire was derived from the software ergonomic standard DIN EN ISO

36 A Didactics-Aware Approach to E-Learning Finally, Fraunhofer IESE designed and guided the cost-benefit analysis. Costbenefit data was collected by DaimlerChrysler and Highware with the help of specifically designed data collection forms. The cost-benefit analysis was based on a 3-phase learning reference model. The purpose of this model is to provide a common basis for comparing different learning and training approaches in one common framework. The reference model consists of the following main phases: Pre-Learning Phase: comprising all relevant activities before learning is performed. Learning Phase: comprising all relevant activities during learning. Post-Learning Phase: comprising all activities taking place after learning is finished. For each of the phases, during the evaluation studies conducted by the application partners DaimlerChrysler and Highware, associated cost and benefit data was collected Evaluation Results in Corporate Environment The first evaluation goal focused on the effectiveness of learning with WBT- Master from the perspective of software organizations, software engineering research organization, and software training service providers, i.e., Daimler- Chrysler, Fraunhofer IESE, and Highware. In accordance with the findings related to the first goal, all partners appreciated the innovative concepts offered by WBT- Master. Nevertheless, the results of the evaluation studies related to learning effectiveness were not fully consistent. The data reported by scientists, software engineers, and software trainers at Fraunhofer IESE, Highware, and Highware s partner and customer organizations generally indicated improved learning effectiveness when using WBT-Master. The analysis of learning effectiveness conducted by DaimlerChrysler was partly influenced by negative judgment of the usability of the WBT-Master platform. This was reflected by the data received from DaimlerChrysler system users who expressed the feeling that the cognitive load associated with tool usage prevented them from learning in the proper sense. In addition to the evaluation studies conducted by DaimlerChrysler, Fraunhofer IESE and Highware within the scope of the CORONET project, a large number of students (more than 100) at the Graz University of Technology have been using WBT-Master extensively since mid-2001 without major problems, thus confirming that the system can be considered a helpful instrument for collaborative learning and knowledge sharing, see sect As a by-product of the analysis of learning effectiveness, some observations and conclusions on cultural and organizational aspects could be drawn from the associated evaluation studies. The analysis of WBT-Master user profiles clearly

37 A Didactics-Aware Approach to E-Learning showed that there was a positive predisposition to work with a web-based learning environment as most of the users had been familiar with ICT for more than two years. However, some cultural factors were detected as being critical. They should be taken into account when introducing and operating WBT-Master. First, shifting to E-Learning clearly requests changes in the behavior of nearly all the participants involved. The changes are mainly related to: Learning approach: shifting from the conventional presence learning mode to using the Internet is not obvious for learners who have not yet had experience with or have not been prepared for using the new learning and knowledge transfer processes offered by a web-based learning environment. Pedagogical approach: replacing interpersonal relations which typically occur in conventional classroom settings by interactions between the learner and the web-based learning environment requires new competence on the part of trainers, tutors, and authors of learning materials. Using a learning environment like WBT-Master is not a one-shot experience: it is highly recommended to properly introduce both the methodology and the infrastructure to all types of users in order to facilitate the adequate use of the learning environment. It clearly appeared from all evaluation studies that system users need some time to handle the new environment before focusing on any specific learning activity. Here are some highlights from the evaluation at DaimlerChrysler. In total, forty individuals were involved in DaimlerChrysler s evaluation studies. Twelve of them actively participated in thirty-four in-depth evaluation sessions. The qualitative analysis applied to the think-aloud protocols and recorded video tapes of the evaluation sessions indicated that the concepts offered by WBT- Master (e.g., to combine collaboration and document work) were generally appreciated by system users. The following functionalities were considered most beneficial for the specific setting of DaimlerChrysler s evaluation study: Inference enabled ontology for self-paced worker s knowledge mining, Web-based tutoring for the experts to give advice, and Various collaboration tools, i.e., forums, to collaborative problem-solving with peer learners, and collaborative knowledge building. The positive impression, however, was negatively influenced by the subjective perception that the current version of the learning prototype platform WBT- Master was too difficult to use. In particular, the various options for communication/collaboration were perceived as too numerous, too spread out, and too hard to differentiate. One possible explanation for these partly negative results is that DaimlerChrysler s software engineers have to cope with extremely high pressure to continuously upgrade their knowledge on-the-job, possibly without

38 A Didactics-Aware Approach to E-Learning being able to spend any effort other than the project-related effort on learning. Hence, this highly specialized clientele is used to (and needs to) work with a software environment that perfectly matches their specific expectations and does not require any introduction and learning curve. Thus, the tolerance level regarding expectations and actual behavior of a new learning environment is rather low. This might explain why WBT-Master, having the maturity of a research prototype, had problems to meet the high expectations of DaimlerChrysler s trial users. Some highlights of the Fraunhofer IESE s evaluation are as follows. In total, seven individuals actively participated in Fraunhofer IESE s evaluation studies. The results of the evaluation studies show that knowledge sharing activities in a research department setting can be performed more efficiently and more effectively with using the knowledge transfer functionality offered by WBT- Master. Last but not least, some of the results of the evaluation at Highware are as follows. In total, thirty-two individuals actively participated in Highware s evaluation studies. The results of the evaluation studies were in its majority positive. The main findings are: The concepts contained in the learning methodology of WBT-Master are presented in a clear and concise style so that learners, trainers, tutors, and authors can easily identify the right learning scenario for their particular learning/training needs. This was particularly true for the scenario Webbased tutoring, which was the main focus of Highware s evaluation studies. The Web-based tutoring functionality provided by WBT-Master offers a viable alternative to classical in-class training settings. The effectiveness of virtual classes was judged as being at least as effective as conventional in-class sessions. Regarding the effective support of web-based experience sharing, the second focus of Highware s evaluation studies, WBT-Master successfully helped to establish a network of geographically distributed learners. From the point of view of the management, the establishment of such a network, facilitating learning at the workplace by connecting people to a network of distributed learning resources (documents, courseware, peers and experts) was considered as one of the main strengths of the CORONET system. Since the main focus of the CORONET project was to develop innovative solutions to support collaborative web-based learning, it was not surprising that evaluation results judged WBT-Master as being acceptable as a training management system, but several proposals for future enhancements were made. It was interesting to observe that the results of the evaluation studies conducted within Highware and in collaboration with Highware s partner and customer

39 A Didactics-Aware Approach to E-Learning organizations in real business cases were more positive than the results of DaimlerChrysler, the other industrial partner in the CORONET consortium. One possible explanation is the following: Since the use cases defined by Highware were more focused, relatively less effort for introducing WBT-Master to their own staff and to their customers was needed. This helped to avoid misunderstandings of the concepts and paved the way for better tool acceptance. It also meant that both Highware staff members (who used the system internally) and end users of Highware s partners and customers were more tolerant towards a prototype system that obviously is not yet perfect (and thus imposes an initial learning curve) but delivers innovative functionality. Another explanation might be the differences between organizational cultures involved in DaimlerChrysler s and Highware s evaluation studies. Both DaimlerChrysler and Highware (and its partners and customers) are highly professional and successful in their respective businesses, but the individuals involved in DaimlerChrysler s evaluation sessions mainly work in complex team-oriented organizational settings with a strong product focus, whereas the individuals involved in Highware s evaluation sessions mainly work in small to medium sized network-like organizational settings with a strong service focus. Whether cultural differences induced by the different geopolitical settings of the studies could also account for the different findings was not investigated. Case studies to analyze perceived ease of use (PEU) and perceived usefulness (PU) of WBT-Master were conducted by DaimlerChrysler, Fraunhofer IESE, Fraunhofer IGD, and Highware. The data analysis of the various evaluation studies did not result in a consistent view. While subjective user acceptance of WBT-Master by individuals involved in DaimlerChrysler s evaluation studies turned out to be insufficient, evaluation data provided by WBT-Master users at Fraunhofer IESE, Fraunhofer IGD, and Highware (including their partner and customer organizations) showed positive PEU and PU ratings. An explanation for this inconsistent result could be the different approaches that were chosen to conduct the evaluation studies. DaimlerChrysler based their analysis on very complex use cases that require a relatively high usability of the tool environment in order to avoid that system users resign from trial experiments. Given that WBT-Master is a prototype platform - and not a fully developed product - time constraints coupled with high expectations of the system users and their low level of tolerance towards initial learning curves resulted in low usability ratings. Hence, the partial non-acceptance of the system by DaimlerChrysler users might be perceived as a confirmation of the project intention to develop a usable prototype, but not an off-the-shelf software product. Due to the different nature of their use cases and the associated learning scenarios it was less difficult for the other partners (Highware, Fraunhofer IESE and Fraunhofer IGD) to introduce the system as a working prototype to their respective end users. As a consequence, in their evaluation studies these partners better managed to focus on innovative functionality and to invest into providing additional learning and user support

40 A Didactics-Aware Approach to E-Learning (which would not be needed for a software product). This led to the positive results of the usability studies of these partners, reconfirming the overall project success. The results of the Cost-Benefit Analysis (CBA) were gained from data collected by DaimlerChrysler and Highware in seven evaluation studies conducted across two evaluation cycles. The majority of the results showed that using WBT-Master is - in addition to the non-monetary benefits generated by the innovative methodology and infrastructure - beneficial from the monetary perspective. The CBA showed that: For Highware, a training provider, using WBT-Master increases the Net Present Value (NPV) and thus can be considered as being monetarily beneficial. For DaimlerChrysler, which is not a training provider, using WBT-Master does not generate a positive NPV in a short term. However, using WBT- Master over a period of more than three years is expected to result in a positive monetary effect. Generalizing the CBA results, it can be expected that: Training providers can be advised to buy and apply WBT-Master as-is because cost savings along with a profit increase caused by travel cost reduction, reuse of training materials, and additional (new) customer services (based on the CORONET features) that generate additional revenue can be expected. Customers of training providers will experience - besides the nonmonetary benefits of CORONET - a cost reduction through reduced interlocation travel of the employees attending to the CORONET-based training. Non-training providers, i.e., software development organizations, can reach a positive NPV in the training and learning cost by using CORONET for a few years Evaluation Results in Academic Environment The project-oriented learning scenario has been applied to conduct the 2002 summer term course on Software Engineering at the Graz University of Technology with more than 200 students. The Software Engineering course at the university consists of: Lectures on basic software development paradigms and vocabularies applied to describe the development paradigms and development processes. Software development project where students develop a software application following one of the presented development methods

41 A Didactics-Aware Approach to E-Learning Thus, the practical part of this course is already project-oriented. Consequently, we wanted to conduct this project by means of WBT-Master. Thus, we prepared a special project-oriented learning session for the Software Engineering project. The session included the following items: Curriculum for the project, where we described the learning goals, learning mode, presented time schedules, etc. A sample software development project clearly identifying the development method, development process, and all steps that students needed to accomplish to successfully finish their projects. Four software development proposals, from which students chose their own projects. The integrated project management tool provided all necessary facilities needed to conduct a Web-based software development project, for both teachers and students. Thus, students made their accounts, groups, and assigned their accounts to the groups. They posted their results as multimedia replies to a particular project folder, following the steps from the sample project. Communicational tools were available for them at any particular time. Teachers were able to track students progress, evaluate the students results and provide them with valuable comments. Discussion forum was used extensively to discuss project related issues among students and among students and teachers. After the course was finished the students and all involved teachers have been provided with a simple evaluation form to evaluate the results of applying this tool in practice. Here are some of the highlights of this evaluation. First of all, there were no additional efforts on the teachers side to prepare and conduct the course. The sample project and the alternatives for students had to be prepared anyway, regardless of the environment where the course was conducted. However, there was a need for a special lecture to explain students how to work with the tool. No other session with students were needed, because all the communication was going on in the online mode. This greatly reduced the time effort on the teachers side because otherwise teachers would need to have 4-5 project meetings with students in the offline mode. The evaluation of students answers was quite positive as well. Firstly, they were asked if accomplishing a Web-based project was more difficult than accomplishing an offline project, which is a project with face-to-face project meetings. Since these students already had a number of projects in other university courses, which were accomplished in the offline mode, their answers might be seen as relevant. Only 5% of students answered that a Web-based project was more difficult to accomplish than similar projects that they had during their classes

42 A Didactics-Aware Approach to E-Learning Secondly, they were asked if they see advantages in using communication and collaboration tools to work together on the project with other students. 80% saw such advantages and stated that the communication using the tool was in the most cases even better than in the offline mode, where the communication is usually restricted to the project meetings. On the question if accomplishing such a Web-based project helped them to acquire additional skills, 90% students answered that they had acquired additional skills, and that there had been no negative difference between the skills acquired as compared with the more traditional projects. 85% of those 90% answered that they acquired these skills because Virtual Project Management Room provided an integrated environment needed to accomplish their task, e.g., they had communication with teachers and other students, possibility to discuss their results, to share their ideas with others, etc. Lastly, they were asked to assess the course and their overall assessment was 1.4, where 1 is the best possible mark on the scale from 1 to 5. The average assessment on the university is 2.5, and the average assessment on our institute is Conclusion and Future Work The evaluation results clearly show that a didactics-aware approach to implementing E-Learning systems and developing standards for E-Learning is a huge step in the right direction. However, some problems related to this approach need still to be resolved. For example, in order to support a new teaching scenario, e.g., a collaborative writing scenario, a new tool must be implemented. Obviously, each new scenario reflecting a particular didactic approach requires such a new tool. This, of course, can cost time and resources. The future work in this direction is going to address this issue in the following way. Firstly, a modeling language for defining different didactic approaches should be developed. With such a language it should be possible to define all the components of a particular teaching scenario, such as educational material, user roles, and the story of the scenario, student activities and others. For example, the story of a particular scenario might be defined as a number of learning actions that students need to accomplish. Student activities might include reading, writing, making tests, and others. Recently, some research efforts were undertaken trying to model didactics from different perspectives, such as constructivist perspective, activity-oriented perspective, etc. [Packer and Goicoechea 2000; Stutt and Motta 2004]. Clearly, such research efforts need to be investigated and as much results as possible coming from that research should be reused. Furthermore, the modeling language should be kept interoperable with the recent E-Learning standards. This will

43 A Didactics-Aware Approach to E-Learning insure that standard compliant educational material can be easily incorporated within the system. Secondly, a single generic tool capable of interpreting and executing teaching scenarios defined by means of the developed modeling language will be implemented into the WBT-Master. This tool will provide an integrated learning environment enclosing all educational material and other WBT-Master tools needed to support a particular scenario. Thirdly, a number of typical teaching scenarios (such as scenarios presented in this chapter) should be modeled by the developed language and executed within the generic tool. Typically, different scenarios will share some common aspects. For example, communication in many different scenarios is usually based on a discussion forum and a chat room. Obviously, these two tools can be coupled together into a single communication component which may be reused as a module in different teaching scenarios. Thus, the modeling language must be component-oriented so that new teaching scenarios might be easily modeled by simply combining a number of already existing components. Lastly, a number of new teaching scenarios should be implemented. These scenarios might include nut should not be limited to scenarios such as collaborative writing, collaborative problem-solving, and others by combining and configuring existing and new components to meet the requirements of a particular teaching scenario

44 3 Learning Scenarios in E-Learning This chapter has been submitted to the review process for the Journal of Universal Science and Technology of Learning. Nowadays, advanced E-Learning systems are generally didactics-aware. Commonly, those systems include facilities for defining so-called learning scenarios that reflect sophisticated didactical approaches such as collaborative writing or project-oriented learning. To support different learning activities from such scenarios the technological infrastructure of these systems must be appropriately adjusted and configured. Usually, this configuration process is related with a number of difficulties. Most of these difficulties are caused by the fact that scenario capturing is achieved through informal user-developer dialogues. Typically, the result of such informal dialogues contains inconsistent and incomplete information because of misunderstandings and the complexity of the interactions within a scenario. Consequently, the configuration of the system is suboptimal and a number of iterations are required in order to achieve better results. In this chapter an approach to improve this situation is presented. This approach is based on a general formal representation model for describing learning scenarios. A particular formal description of a concrete learning scenario is obtained through a user dialogue with a wizard tool. At the next step, this formal description might be automatically processed to facilitate configuration process. The chapter is concluded with some experiences gained by applying this approach in two E- Learning projects. 3.1 Introduction Successful E-Learning projects always concentrate on didactical aspects to enable efficient technology-enhanced knowledge transfer. As such, these projects are far less focused on the technology itself [Hirumi 2002; Hobbs, 2002; King and

45 Learning Scenarios in E-Learning Puntambekar, 2003; Leasure et al., 2000; Mioduser et al., 2000; Oliver et al., 2002]. For example, one of such projects was the project called CORONET (Corporate Software Engineering Knowledge Networks for Improved Training of the Work Force) that was funded by the European Commission (IST ). The main purpose of the project was to analyze, implement and evaluate a number of tools for support of collaborative knowledge transfer processes. Each of such tools utilized the current and advanced Web technology to facilitate and speed the flow of knowledge from people possessing that knowledge to people who needed to acquire it by following a particular collaborative didactical approach. Thus, processes such as Web-based tutoring, Web-based knowledge mining, Web-based collaborative writing, and collaborative project-oriented learning have been supported. The evaluation of the project results in respect to the increase of learning effectiveness by knowledge sharing and collaborative learning generally indicated improved learning effectiveness [Pfahl et al., 2004; Helic et al., 2004; Helic et al., 2005]. One simple strategy to utilize didactical aspects in E-Learning is based on managing so-called learning scenarios. This strategy was successfully applied in a number of E-Learning projects at the University of Technology Graz [Helic, 2006; Helic and Durco, 2005; Ebner et al., 2005, Dreher et al., 2004]. In those projects a learning scenario was defined as a combination of the following components: A particular way (i.e. a story) of working with the system to achieve a particular learning goal. Typically, the story was represented as a collection of learning activities that need to be carried out to accomplish the goal. The user roles that are involved in the story, e.g. teachers, tutors, students, or learners. The system tools, features and services that are needed to support the activities. The educational content relevant to the learning goal. Using this scenario structure, scenarios such as thematic uploading, thematic discussion, goal-oriented, or reflective learning were developed in those projects. Further, one of the important requirements in these projects was a possibility to introduce new learning scenarios and customize already existing ones. The systems and technological infrastructure fulfilled that requirement by providing a generic learning scenario framework. The framework utilized extensibility and customizability by providing flexible system configuration possibilities. Thus, supporting a new learning scenario or customizing an already existing one was equivalent to providing a new system configuration or adjusting an existing configuration, respectively

46 Learning Scenarios in E-Learning However, such a general and flexible didactics-aware framework is inevitably technically complex. Therefore, the process of configuring the framework to reflect the scenarios is related with a number of problems. Typically, these problems are visible at different stages of such a scenario management process. Capturing of scenarios is tedious and error-prone. It is carried out in informal settings through user-developer dialogues. The results of such dialogues are scenario requirements documents in a narrative form. Typically, this final result contains inconsistent and incomplete information because of misunderstandings or the complexity of the interactions within the scenario. Basically, this problem is equivalent to a general user requirements analysis problem in any software and system development process [Meyer, 1985; Jarke et al., 1998]. Technical realization, i.e. configuration, and customization are extremely difficult because of the rich and complex functionality that is needed and because of the heterogeneity of the technological environment in question. Firstly, in order to support sophisticated learning activities and scenarios the system must provide sophisticated functionality. That functionality is typically organized in highly complex structures and processes. For example, collaborative writing scenario includes activities such as writing, reading, reflecting, commenting, and discussion. These activities can be mapped on the system functionality in the following way. The discussion activity is mapped onto a discussion forum. The writing activity is mapped onto an editor tool with upload functionality. Additionally, the editor is associated with a version control system to control different versions of the documents. The reflecting and commenting activities might be mapped onto an annotation tool, and all of the activities are connected to the user authentication and user rights module that decide who can do what. Thus, even this simple scenario example can lead to a complex development and configuration process. Moreover, E-Learning systems are typically developed in a Web-based technological environment, which is a very fast evolving and everchanging environment. Obviously, such a dynamic environment increases the complexity of the development and configuration process of E-Learning systems. Technically, one possibility to remedy these problems is by developing formal specifications of learning scenarios. Such formal specifications would meet two goals; firstly, inconsistency and incompleteness of a captured scenario description can be partially or completely avoided; secondly, strategies and automatic or semi-automatic procedures for supporting system configuration and customization can be developed on the top of that formalism. As a consequence of these two goals formal specifications must be: Easy to understand and use for all, i.e., the users of the system (e.g., teachers, tutors, students, etc.) as well as the developers of the system. Sufficiently expressive, yet still simple enough as to allow semi-automatic or automatic processing and analysis in order to obtain at least a first

47 Learning Scenarios in E-Learning prototype of the system at different levels of abstraction and granularity, e.g., architectural model, component-model, or even automatically generated configuration scripts or implementation and integration code if needed. The idea of formal specification of the user requirements (user scenarios, use cases) is by no means a new idea in software engineering. For many years now the developers and researchers alike have been trying to apply formal methods in software development in general and user requirements engineering in particular [Meyer, 1985; Clarke and Wing, 1996, Hong and Lingzi, 2000]. However, there exist a number of difficulties related to formal methods in developing information systems caused by fast changing requirements, user bases, usability and design issues, or rapidly changing technologies [Land and Hirschheim, 1983; Avgerou, 1987; Overmyer, 2000; Bolchini and Paolini 2004]. Moreover, such formal methods are typically applied only for validation and checking purposes rather than as a system configuration support [German, 2000]. One of the possibilities to reduce the complexity of these problems is to use domain-specific knowledge in developing a formal representation model. The domain knowledge can be used not only to support capturing of the user and scenario requirements but also for automatic or semi-automatic processing and analysis of the collected scenarios by utilizing the domain-specific semantics. This approach is very similar to a modern software engineering approach called Model Driven Architecture (MDA) first proposed by the Object Management Group (OMG). According to this design approach a system specification is developed in the form of a formal domain model that can be processed to automatically generate parts of implementation, integration, or system configuration code [OMG, 2003]. 3.2 Management of Learning Scenarios One of the basic problems in capturing of scenarios is a mutual lack of knowledge about the technical and didactical aspects on the side of the users (teachers) and system developers, respectively. On the one hand, the users of the systems lack the system expertise and experience, i.e. they have a particular didactical approach in mind, but they do not know if and how that didactical approach or at least part of it is technically feasible. On the other hand, the system developers have limited understanding of the domain, the subject matter, and the didactics involved in a learning scenario. Thus, there exists a so-called impedance mismatch between the didactical aspects of a learning scenario and its technical realization. In other words, both the system developers and users are confronted with input from the other side that is typically inadequate to them. Let us illustrate this problem with two real-life examples from some of our recent E-Learning projects

48 Learning Scenarios in E-Learning Problems with Scenario Management: Case Study 1 The first project in question is Ephras [Helic, 2006; Helic and Durco, 2005], a project that has been funded by the European Commission under the Socrates/Lingua2 programme ( CP SI-LINGUA-L2). The goal of the project was to develop a computer supported phraseology learning material for students of foreign languages for four European languages - German, Slovak, Slovenian and Hungarian language. One of the E-Learning components developed within the project was a module with 150 interactive tests to selected phrases in those four languages. Those tests served to check understanding and knowledge of the phrases in question, as well as for improvement of skills for producing these phrases in a written and spoken foreign language. As such they followed a sound didactic approach that has been developed by the didactical partners (foreign language teachers) in the form of an exercise typology. This exercise typology was a result of expertise and experience of the teachers collected in a number of years in teaching phraseology in traditional classroom settings. Additionally, the questions and tests have been aligned according to language, topic, knowledge, and skill level of the students. On the technical side, IMS Question & Test Interoperability (QTI) [IMS QTI, 2005] specification has been applied to develop the exercise module. The QTI is a de facto standard for cross-platform representation of questions and tests in Learning Management Systems (LMS) and there are freely available tools for authoring, running, or result processing for QTI tests. While developing the exercise module we have experienced the following problems caused by the above mentioned impedance mismatch between the QTI standard and its concrete application in a specific subject area such as foreign language teaching. The mismatch is visible at two levels. First, the QTI standard is a technical specification which supports development of questions and tests for various subject domains where the questions and tests are interoperable at the level of authoring tools, question/test databases or LMS. As such the QTI standard defines a number of general question types that might be applied in a number of areas and does not take into account specific question and test types of a particular domain [Milligan, 2003]. However, there exist a number of specific question types which are commonly used in foreign language teaching but are not reflected in the QTI standard, with crossword puzzles being only one typical example. Second, the QTI standard is not concerned with didactical issues and, as a matter of fact, it tries to be as didactically neutral as possible [Smythe and Roberts, 2000]. Yet the basis for development of the interactive tests in the Ephras project is a sound and successful didactical approach for teaching phrases in foreign languages. This approach includes learning activities such as phrase recognition,

49 Learning Scenarios in E-Learning phrase meaning or phrase pragmatics identification, phrase form and grammar understanding, consolidation and reflection. Each of these activities is typically represented by a number of exercise types which are used in accordance with the current context, as well as the student s knowledge and skill level. For example, identifying phrase meaning can be realized with a multiple choice question where the student needs to select the correct meaning of a phrase from a number of possible answers, with a short text essay where the student needs to write down the meaning of the phrase, with a combination of these two question types, or with a simple drag-and-drop exercise where the student needs to correlate the phrase meaning with a particular graphical representation of that phrase (see Figure 3.1). Figure 3.1: Drag-and-drop exercise to support phrase meaning identification Basically, development of scenarios was an iterative process with a number of repeating steps. The whole process required collaborative efforts on the side of the system developers and the teachers. The iteration steps of this development process included: Initial discussion between the system developers and teachers. The main goal of this session was to achieve a better understanding of both the didactical requirements for questions and tests and the technical possibilities offered by the QTI standard. Thus, the teachers explained the types of questions and tests that are needed, whereas the system developers explained the features of the QTI standard. Preparation of several QTI samples to illustrate the possibilities of the QTI standard. The prepared samples already dealt with the questions and tests from the subject matter. Agreement on a text-based informal format for defining the questions and tests by the teachers. Such an informal format was needed because the teachers were not familiar with creation of formal specifications, i.e. they had a non-technical background. Realization of scenarios by means of QTI standard

50 Learning Scenarios in E-Learning Test and improvement phase. Note that some of the steps have been repeated a couple of times to obtain optimal results. Thus, the sheer amount of the development steps, as well as the amount of the work needed for communication, testing or improvement resulted in a very tedious work. Moreover, because of misunderstandings, communication problems, or implementation difficulties the whole development process was error-prone. The difficulties of the scenario capturing process and their technical realization can be summarized as follows: Learning curve for the QTI standard for the teachers was very steep. There are no sufficient manuals, help files, tutorials or tools that would decrease the time needed to learn the standard. Informal nature of format for defining the needed scenarios leads to misunderstanding problems between the teachers themselves. The consequences of such problems become even more serious with an increasing number of teachers, because format disparity between different teachers also increases. Realization of scenarios using such an informal format is also very difficult because of misunderstanding problems between the authors and the systems developers. Again, the disadvantages become more visible as the number of teachers increases Problems with Scenario Management: Case Study 2 The second project in question is ivisice (interactive Visualizations in Civil Engineering), an E-Learning project to support the study of Civil Engineering at Graz University of Technology [Ebner and Holzinger, 2002]. Originally, the aim of the project was to investigate the possibilities of applying Web technology in education of structural engineering. Due to the fact that civil engineering students need to obtain intuitive understanding of structural behaviour the didactical approach is strongly based on visualizations. In addition, communication and interaction among all participants complete the learning scenario. Consequently, a great number of Web based animations, visualizations and interactive learning objects have been developed to visualize and to simulate highly complex processes [Ebner and Holzinger, 2003]. This content was implemented in a course management system, which has been made accessible by the computing department of the University of Technology Graz. The combination seemed to be quite successful for the first time. Subsequently, the gap between the sophisticated and up-to-date content [Holzinger and Ebner, 2003] and an obsolete, rigid content delivering platform became bigger and led to a dissatisfaction of end users, teachers and students alike. Finally, the course support team decided to follow a new paradigm where the content delivering system can be adapted and upgraded in the same way as content complexity and user engagement are growing up. For

51 Learning Scenarios in E-Learning that purpose a novel E-Learning platform called WBT-Master, developed at the University of Technology Graz, has been chosen [Ebner et al, 2005]. To fulfill the requirements of the ivisice project a WBT-Master component for management of so-called training objects has been applied ( [Helic et al., 2004]. A typical example of such training objects is a simple learning unit (i.e. a number of documents combined into a reusable, navigable collection), but this is only a single simple type of available training objects. Discussion forums, chats, quizzes, virtual laboratories, or project management rooms are also training objects which can be managed in WBT- Master. Each training object implements a particular training paradigm. For example, students are supposed to answer questions if they use an "examination room"; upload and discuss reports, if they use a "thematic discussion"; follow online/offline step-by-step explanations of most difficult topics in a tutoring session or in a mentoring session ; or develop a project in collaboration with colleagues, if they use a "project management room". Additionally, training objects can be combined into a new single entity called "training course". A training course is just a combination of training objects selected for a particular study that provides also some additional communication, collaboration, and user administration tools such as evaluation, or grading tools. If we compare WBT-Master with existing E-Learning solutions one essential advantage can be recognized and that is the system s flexibility and customizability: these features are just as desirable for any other E-Learning solution but they are an inherent part of the WBT-Master training paradigm. Any teacher selects just a few components and combines them into a new training course to provide a required training curriculum and system functionality [Ebner et al., 2005]. The new didactical paradigm in the ivisice project supported by means of WBT- Master was based on the assumption that although learning is an active cognitive process on the part of the learner, it is also a social process and develops through conversation [Motschnig-Pitrik and Holzinger, 2002]. Therefore, interaction, participation and communication have been recognized as crucial elements for ivisice, in which the learning environment has to enforce the possibility of community building. It should be pointed out, that the aim of this new paradigm was to compensate the lack of communication features. More precisely, the problem was the missing communication tools developed for a specific user group students of Civil Engineering. Thus, a new training course has been developed. Besides content management features and user management facilities this training course combined a number of communication tools such as [Ebner et al., 20005]: A discussion forum for writing contributions in a structured manner. A Web-based whiteboard for synchronous textual and vector graphic based communication

52 Learning Scenarios in E-Learning A thematic upload tool to help students upload, manage and discuss their practical examples for passing the examination. Furthermore, the tool assists the teachers regarding the correct time management, i.e. upload was not possible after a predefined time limit (see Figure 3.2). FAQs for exporting frequent questions from the discussion forum and arranging them into an easily navigable FAQ taxonomy. Again, the development process was iterative and span over a number of terms and student groups with gradual improvements in the didactical approach, but also in the technical infrastructure needed to support that approach. The typical steps of the development process included: Initial discussion between the system developers and teachers, again to achieve a better understanding of both the didactical requirements, as well as technical possibilities offered by WBT-Master. Preparation of a first prototype in the form of a training course to illustrate the possibilities of WBT-Master. Test and improvement phase. Deployment and production phase, where the prototype was constantly improved by adding new training objects, customizing features, or adding new functionality. Figure 3.2: Thematic upload tool for management of student examples Basically, all of these steps have been iterated at all times, with the current result achieved after three school years, two different courses, and a production phase with more than 200 Civil Engineering students. In this project, the difficulties of the scenario capturing and realization process are similar to those experienced in the Ephras project: a steep learning curve for the system in question; informal nature of scenario capturing; and complexity of the didactical approach all lead to misunderstanding problems, incomplete, and inconsistent information

53 Learning Scenarios in E-Learning There is also another difficulty characteristic for the ivisice project. Since the didactical approach used in the project was based on interaction, communication, collaboration, community building, and other social aspects of learning, the learning environment in question exhibited a highly dynamic nature. In such an environment the requirements are never stable and change on a regular basis. As a consequence, the technical infrastructure is in a constant beta state, similar to social applications typical for Web 2.0 platform, and needs to be improved, adjusted, and customized very frequently. This deepens the problem not only for the system developers who need to react to ever-changing user requirements, but also for the teachers who manage the student community, and need to provide almost immediate feedback to students request. In many cases, especially when the teachers are already experienced users of the system and understand the possibilities offered by the system such problems are easily handled by the teachers themselves. For example, the teachers might demonstrate to the students how to use a particular feature in a proper way, point to a system component which might be used in a particular learning situation, or ask the system developers to extend the system functionality if there is a need for this. However, in the situation where the teachers do not posses sufficient system expertise (i.e. in the initial learning sessions) this can lead to serious problems. On the one hand, the teachers might reject an excellent student idea for improvement of the learning environment because they do not believe that the idea is technically feasible. On the other hand, they might accept an improvement suggestion for which the technical realization might be very difficult or even not possible at all Problems with Scenario Management: Conclusions In both of the case studies scenario management process was related with a number of difficulties. Besides the common issues related to the user requirements engineering such as misunderstandings, incomplete and inconsistent information, or complexity of the domain (e.g. sophisticated didactical approaches), the dynamics of a socially-aware collaborative learning environment in the second case study introduced another level of complexity in the process. In both cases, such issues could be only resolved through an iterative development and management process over a longer period of time. Thereby, at each new iteration step a new prototype has been introduced that represented a slightly closer approximation of what the users really wanted. These two case studies demonstrate very clearly that scenario management in E- Learning is inherently complex, i.e. the difficulties exist at all levels of abstraction or granularity of a learning scenario. For example, from the didactic point of view in the ivisice project we have been working on a higher abstraction level, i.e. the teachers defined a number of learning activities for their students and the system developers integrated a number of tools into a training course to support those activities. The activities included structured discussion, chat, drawing, or preparing and uploading materials. The teachers did not go into details of these

54 Learning Scenarios in E-Learning activities because in this case the details did not have didactical relevance (except for preparing and uploading of materials where details such as deadlines, format, quantity and quality of material have been defined). On the other hand, in the Ephras project the teachers worked on a lower didactical abstraction level, i.e. they defined a single learning activity (exercises) but with a finer granularity. However, in both cases the difficulties have been present in capturing and technically realizing learning scenarios. Also, complexity of the management process was inescapable in both cases. Consequently, if we want to deal with more complex learning scenarios that work at different didactical abstraction levels with a fine granularity at each of these levels the overall complexity of the learning scenario management process grows. For example, suppose that in the ivisice project the teachers wanted to conduct an online examination using a question-test module from the system. For instance, the examinations could follow a common examination strategy from traditional settings in Civil Engineering education and include multiple choice questions, fillin-blank, free essays, calculus problems, or drawing and designing engineering solutions. Each of these question types might require additional properties to be defined, e.g. for a fill-in-blank question the teachers might need to define how many blanks exist or how many positions each blank has. Obviously, capturing and realizing of these scenario requirements is very similar to the Ephras project, i.e. we need to deal once more with the same difficulties at this lower level of abstraction. Such difficulties are by no means new in software engineering field and have been recognized previously. For example, Brooks states that the complexity of software is an essential property, not an accidental one [Brooks 1987]. As Booch points out the main reason for that complexity is the complexity of the problem domain [Booch 1993]. Obviously, such a complexity is inherent in didactics-aware E-Learning systems. A common strategy for tackling such difficulties in complex didactical scenarios, as well as in software development is to hide irrelevant details by working at an appropriate abstraction level and only switch to the next abstraction level or to a finer level of granularity when there is a need for it. Therefore, each technical solution for the scenario capturing and realization problem needs to follow a similar strategy, i.e. it needs to provide a mechanism for hiding of irrelevant details as well as a mechanism for switching between different abstraction and granularity levels. 3.3 Representation of Learning Scenarios As a solution to the above mentioned difficulties a formal representation model has been developed. The representation model comprises three components: a meta-model that defines the modeling elements and their semantics; domain models that represent particular domains and that are developed using the elements of the meta-model; a formal binding that defines an XML binding to

55 Learning Scenarios in E-Learning export domain models. On top of this representation two tools have been developed: a GUI tool that supports capturing of a particular learning scenario by displaying the elements of a domain model in a user-friendly manner and an automatic processing component that processes captured learning scenarios to infer a corresponding system configuration. Both of these components work with XML-based representations (that are valid XML documents in respect to the defined XML binding) of domain models and learning scenarios Meta-model The above discussion shows that the scenario capturing and realization difficulties exist at different levels of abstraction and granularity. Thus, the first requirement for the meta-model is a possibility to represent these different abstraction and granularity levels. Furthermore, to reduce the complexity of a domain model it should be possible to hide irrelevant details of that model as the need arises. Object-oriented representation seems to be a perfect fit for such a meta-model because it is a general-purpose domain representation mechanism. The basic elements of object-orientation include objects, attributes, classes, instantiation, encapsulation and details hiding, structural relations (i.e. specialization, generalization and composition), inheritance, and associations. Let us look now at these basic modeling elements in details to investigate how they fit into the above defined requirements. Objects are used to represent concept or real-life entities. Objects typically have one or more attributes that represent the current state of an object. Similar objects belong to a class of objects that defines all attributes that these similar objects might have. Usually, we speak of objects as instances of a particular class that encapsulate and hide their current state in the form of attributes. The instantiation property is typically represented by means of an is-a relation. The encapsulation and hiding mechanism is a very important concept that allows us to hide irrelevant information from the rest of a domain model and only dive into the details if there is a need for it. In addition, objects and classes are typically related with other objects and classes. With structural relations we can express domain knowledge with differing levels of abstraction and details. There are three typical structural relations in objectoriented modeling: specialization, generalization (which can be comprehended as an inverse relation of specialization), and composition. Specialization reflects a so-called is-a-kind-of relation between two domain classes and states that a particular class is a special case of another class. For example, the QTI standard defines questions of type multiple-choice - it is a question type where students can select one or more answers from a number of answers to a particular question. Further, the standard defines a number of special

56 Learning Scenarios in E-Learning cases of multiple-choice questions regarding the format and type of the answers there can be only textual answers (text multiple-choice), image answers (image multiple-choice), or hotspot multiple-choice where students can select a particular area of an image as an answer. Composition reflects a so-called is-a-part-of ( part-whole ) relation between two objects. For instance, the QTI standard defines a notion of a test or a quiz, which is a collection of questions of different types, i.e. the questions are related with an is-a-part-of relation with the quiz. Through the mechanism of inheritance we can further utilize specialization relation. Inheritance means that a special case of a class inherits all the properties (attributes and relations) of the general class. Further, the special case can refine and redefine these properties if needed. For example, typical attributes of a multiple-choice question include the question, the number of possible answers, the number of correct answers, the number of answers that students might select, and the answers themselves. One special case of a multiple-choice question is a standard true/false question, where all attributes of a general multiple-choice are inherited but the special case sets the values for those attributes by a definition. Thus, the number of possible answers is two, the number of correct answers is one, the number of answers that students might select is also one, and the answers are true and false. The structural relations, i.e. specialization and composition and the inheritance mechanism allow us to create domain models at different abstraction and granularity levels. We can utilize these different levels by means of a clever user-interface that will support users in working on a single level of abstraction or granularity, and only going into details (a lower level of abstraction or a finer granularity level) when there is a need to describe these details as well. Such a user-interface would reduce the complexity of scenario capturing process to a great extent. By means of associations we might express relations between objects and classes across the whole model, i.e. across the different levels of abstraction, across different levels of granularity, as well as across compositions. Let us here investigate shortly an example from WBT-Master. Suppose that a special training course should be created supporting collaborative writing didactical approach. In a typical collaborative writing scenario students write and comment on a series of documents. The teacher might provide a feedback for the students. In the next step, the learners work on improving the documents. At the highest abstraction level (at the didactical level) a model of scenario might include the following student activities: writing, reading, commenting, reading feedback, improving results. However, we still need a technical infrastructure that will support these activities in a learning environment. Thus, that technical infrastructure needs to be reflected in the model and related to the didactical abstraction level. Obviously, the model of the technical infrastructure is on another abstraction level and we

57 Learning Scenarios in E-Learning need to relate objects and classes between these two abstraction levels to realize the connection between them. For that purpose we might use associations. For example, we might introduce a supports association to model the fact that a particular tool supports an activity from the didactical level. Thus, we might say that a document viewer supports reading activity; a document editor supports writing activity; or an annotation tool supports commenting activity. In addition to associations between objects and classes from different abstraction levels, the objects and classes from one and the same abstraction level might be associated with each other. For example, the document editor that supports the writing part of the collaborative writing needs to deal with different versions of the documents. Typically, the editor communicates with an external version control tool to manage the document versions. Thus, we can say that the document editor depends on the version control tool. Note here, that such associations are of crucial importance for the automatic processing phase where associations might be processed in a number of ways, e.g. to automatically infer the tools that are needed, to highlight dependencies between the tools, or simply to obtain configuration settings that might support a particular learning scenario. For a better understanding object-oriented domain models are commonly represented in a graphical way. For simplicity we use simple directed labeled graphs to represent object-oriented models of learning scenarios with the following definition of graph semantics. All the objects and classes regardless of abstraction level, as well as all attribute values are represented as graph nodes. The proper abstraction level and the distinction between the objects, classes, and attributes are typically visible from the context of a node. All the structural relations, associations, and attribute names are represented as directed labeled edges between the nodes. Again, the distinction between different edges and their semantics is typically trivial and visible from a wider context of an edge. Figure 3.3 shows an example of a domain model of QTI that represents quiz, multiplechoice question and true/false question (a specialization of multiple-choice question)

58 Learning Scenarios in E-Learning Figure 3.3: Graphical representation of a part of a QTI domain model Domain Models Using the presented object-oriented meta-model we can now develop a number of domain models. The domain model development consists of identifying the key classes, their attributes, and relations with other classes in a particular domain. Note here that such domain models can be at different levels of granularity, i.e. one domain model can be a detailed model of a single class from another model. The possibility to hierarchically structure different abstraction and granularity levels provided by the meta-model makes it highly expressive. However, through the mechanism of encapsulation and information hiding the domain models are kept simple this allows a possibility to provide a mechanism for automatic processing of such domain models. Domain Model: QTI The model of the QTI standard is a rather simple one. Note here that we will not deal with all the features of QTI because that is, obviously, out of scope of this paper. Rather, a simple conceptual model of the QTI standard will be developed only to illustrate modeling capabilities of the object-oriented meta-model. Thus, in the QTI standard we might identify the following classes: Question is a basic unit of the QTI standard. Each question has the text of the question, and might have information on response processing, i.e. feedback for students in the case that the answer was correct, feedback for students in the case that the answer was wrong, or the number of points that students get for a correct answer. Quiz (Test) is a collection of a number of question instances

59 Learning Scenarios in E-Learning Multiple-Choice question is a specialization of the question class that requires from students to choose one or more correct answers from a number of answers. There are several subtypes of the general multiplechoice question such as single correct answer, true/false question, or multiple correct answers. These subtypes provide different default values for some of the attributes of the general concept. Moreover, different types might be identified regarding the format of answers such as textual answers, image answers, image hotspot answers, or mixed answers. Fill-in-blank question is another specialization of the question class where students need to type in a correct answer into allocated space in the form of a text. Depending on how the text is interpreted different subtypes of the general concept can be identified. For example, the QTI standard defines the following interpretations: string, integer, and decimal. A special case of the fill-in-blank question is a so-called short essay which defines an alternative way of result processing, namely it allows students to enter free text, and therefore only provides hints for students as a response. Drag and drop question where students need to drag objects with mouse to appropriate positions on the screen. Again, there are several subtypes of this question depending of the kind of the objects that can be dragged (textual objects or images). Also, depending on the spatial distribution of objects and spatial direction in which the objects can be dragged a number of subtypes might be identified. For example, sometimes the objects might be dragged only in horizontal or vertical direction making that question a simple objects ordering exercise. Note here that the QTI standard includes additional question types and a more detailed specification of their attributes, or result processing possibilities. However, with the simple model from above and the capabilities of objectoriented modeling discussed above (i.e. composition, inheritance, and encapsulation) we have been able to create very sophisticated models of QTI quizzes and questions that completely covered scenario requirements in the Ephras project. Domain Model: WBT-Master The model of WBT-Master is a rather complex one, since WBT-Master is a fullyfledged E-Learning system that is based on a wide range of didactical and pedagogical concepts realized with sophisticated and complex technical infrastructure. For example, the question and test module that is based on the QTI standard is only a single component of WBT-Master. However, as mentioned above inheritance and encapsulation might be used to represent the hierarchical nature of such a system and to hide irrelevant details from a model. Here we will describe only two higher abstraction levels of WBT-Master and will not go into details of different modules

60 Learning Scenarios in E-Learning WBT-Master is based on a sound didactical approach, i.e. it is based on collaborative, socio-constructivist, and activity-oriented learning theory. As such it supports students and teachers in a wide range of activities that are centered on a community building process. This didactical level is the highest abstraction level of WBT-Master. It includes the following classes: User role refers to different didactical roles that users might have in a learning scenario, e.g. there are students, tutors, trainers, teachers, or lecturers. Each of these roles is modeled as a specialization of the user role concept. Activity is a single task that is executed by a user with a specific role. There exist a wide range of activities, each of them reflecting a particular collaborative learning task. For example, there are activities such as reading, writing, uploading, testing (recollect that if there is a need for detailed modeling of this activity we might use the QTI model from above similar models might be developed for other activities as well), or communicating. Some of these activities have subtypes, e.g. communicating might be discussion, commenting, providing feedback, chatting, collaborative drawing, and similar. Activity pattern is a composite activity that might include a number of other activities in a single new activity. For example, collaborative writing is such a composite activity that has as its components writing, reading, reflecting, and commenting activities. At the same abstraction level we can identify a so-called learning environment, but we will look onto it separately for comprehension purposes. The learning environment contains the following classes: Training (learning) object which is a basic unit of learning material or content. Typically, each activity is associated with one or more training objects, e.g. a reading activity is associated with a document. Training course provides a particular learning context by enclosing a number of activities and training objects into a coherent learning entity. In addition, a training course might provide a training curriculum, additional communication features to enhance community building process (e.g. user awareness features, group management features, and similar). Lastly, let us define classes that are related to the technological infrastructure of the system. Note here that the classes from this abstraction level are typically related with the classes from didactical level by means of associations, e.g. by means of the supports association. Some of the technological classes include: Training object library which is a collection of a number of training objects that supports creation of training courses. Training object editor that supports authoring, editing, and managing of training objects

61 Learning Scenarios in E-Learning Discussion forum which supports asynchronous moderated or nonmoderated discussion and communication. Annotation tool which supports commenting and feedback activity. Assessment tool which supports grading feedback activity. which supports asynchronous exchange of messages. Instant messaging tool which supports synchronous exchange of messages. Chat tool which supports synchronous communication. Whiteboard tool which supports synchronous communication enhanced with possibilities to exchange images, vector graphics, or to draw in a synchronous manner. Quiz/test module that supports authoring, management, and execution of questions, quizzes and tests. Upload tool to support publishing of student results. Editor tool that supports writing and editing of documents. User management tool that utilizes authentication facilities and user rights management. Typically, all other tool and modules depend on the user management tool. Note here that the list of technological classes does not raise a claim on completeness rather it tries to serve only as an illustrative example of a domain model of the technological infrastructure of an E-Learning system Formal binding: XML To export domain models an XML binding has been developed. The basic XML model is a labeled ordered tree where labels represent node names. Edges are always directed (to preserve the tree order) and do not have labels. Additionally, XML supports a referencing mechanism between nodes, which basically facilitates modeling of arbitrary graphs. In this way directed-labeled graphs might be represented by means of XML documents. An excerpt from an XML document that formally represents hierarchy of multiple-choice question types (see Figure 3.3) is shown in Listing 3.1. Note also the representation of attributes. <model> <name>qti</name> <class id="1"> <name>question</name> <attribute key="quest_text"type="string"/> </class> <class id="11"> <is-a-kind-of refid="1"/> <name>multiple Choice</name> <attribute key="answer_count" type="int"/> </class>

62 Learning Scenarios in E-Learning <class id="111"> <is-a-kind-of refid="11"/> <name>true/false (Text)</name> <attribute key="answer_count" default="2"/> </class> </model> Listing 3.1: XML-based representation of multiple-choice questions Through referencing mechanism for XML documents a number of models might be combined. For example, an XML document that represents WBT-Master model might simply refer to the QTI representation to provide a model of its question/test module User Interface In order to simplify capturing of learning scenarios a user-interface module has been implemented on the top of the defined XML binding. The most important requirement for the user-interface module was the usability, i.e. it should be intuitive and easy-to-use. For that purpose the user-interface module has been developed in the form of a wizard. The wizard leads the users in a step-by-step fashion through the model in question. In addition, it allows them to explore different domain classes in more detail by navigating trough the model s hierarchical structure, exploring and learning in this way different model features. For example, the users might retrieve more detailed description of a particular class, or compare that class with other similar classes or with classes from the same abstraction level. Structural relations ( is-a-kind-of and is-a-part-of ) serve as the basis for the navigation direction within the wizard, i.e. the users investigate at each moment only a single class but are provided with links to access components of that class or its specializations (see Figure 3.4). In the cases of multiple inheritances or whenever a class belongs to a number of classes a history stack is utilized to support the users in navigation to the opposite direction. Additionally, at each navigation step the users might select a particular class to include it in a current learning scenario. If a particular class has attributes they are first populated with default and inherited values, and in addition the users might be asked to enter remaining attribute values (see Figure 3.5). Basically, selecting a class for the scenario and populating its attributes with values is equivalent to instantiation in object-oriented sense, i.e. the users create instances of that particular class and define their current state in this way

63 Learning Scenarios in E-Learning Figure 3.4: Wizard-like user interface of the QTI model Figure 3.6 depicts the relation between a particular domain model and a captured learning scenario. The scenario contains a number of objects that are instances of classes defined in the domain model. Figure 3.5: Setting attributes of a multiple-choice question

64 Learning Scenarios in E-Learning Figure 3.6: Relation between a domain model and a captured learning scenario Lastly, after the users finish their exploration session a simple (formal) description of the current learning scenario is obtained. That description lists all of the created class instances together with the values of the associated attributes. The current learning scenario is also represented as an XML document. That document is used as the basis for further automatic processing of the scenario Processing of Scenarios The formal XML-based representation of a scenario obtained during a user exploration session might be automatically processed to support system configuration. There are several strategies for such an automatic processing the choice of an appropriate strategy depends on the degree to which a particular system or module can be dynamically configured or customized. Thus, there are two classes of systems the first one is a class of systems that are dynamically configurable. For example, a QTI quiz is a simple collection of a number of questions. Since the QTI standard defines an XML binding a QTI quiz might be easily represented as a number of valid XML documents. Obviously, transforming an XML document that captures a scenario onto a number of standard QTI documents is a trivial task that can be accomplished by means of different technologies such as XSLT, a simple transformation script, or a special transformation program. Note here, that the obtained QTI XML documents might be used as a certain kind of templates to rapidly produce instances of these questions and quizzes. The templates might be loaded into a standard compatible QTI editor to enter the specific question, answers, or feedback. Figure 3.7 shows

65 Learning Scenarios in E-Learning such a template that has been used in the Ephras project the template is a very simple quiz that combines a short essay and a multiple-choice question. Figure 3.7: Editing of an automatically created QTI quiz The second class is a class of systems that exhibit a total or partial lack of possibility for dynamic configuration or customization. For example, some of the features and functionality in WBT-Master can be easily customized in the form of a new training course. However, there are certain features or modules that are preconfigured and their configuration can not be in any way dynamically or automatically adjusted. For instance, a discussion forum in WBT-Master offers a standard possibility to attach a local file to a posting. There is no technical possibility whatsoever to remove that feature from a discussion forum in an automatic way because that feature is hard-coded in the implementation of the discussion forum. Moreover, WBT-Master offers more than 20 predefined and preconfigured training courses which reflect a wide range of common collaborative didactical approaches such as project-oriented learning, collaborative writing, or brainstorming. Again, certain modules of these training courses are hard-coded and can not be automatically customized. As a matter of fact, the software architecture of WBT-Master does not provide a possibility for a complete dynamic configuration of a training course, i.e. there is always a need to manually adjust configuration scripts or source code to implement a completely new training course. It is our experience that most of the current E-Learning systems exhibit the same behavior, i.e. it is not possible to configure them dynamically in full. Obviously, in such cases a partial solution should be

66 Learning Scenarios in E-Learning achieved. How such a partial solution looks like depends on a particular configuration strategy being applied. Firstly, the system can compare the selections made by user with the predefined configurations and try to estimate a best-match configuration. The estimation can be implemented in a number of ways. For example, a very simple solution would be to base the estimation on a Boolean model, i.e. whenever the user selects a particular scenario feature the system investigates all predefined configuration and keeps only those where the selected feature is contained. At the next step the system marks the remaining features either as selectable or as non-selectable and reflects this via user-interface. The selectable features are only those features that are components of one of the remaining configurations. At the end the users obtain an exact match for their requirements; however, the users are restricted in possibilities what they can select. A slightly better solution might be to use an algebraic or probabilistic model for estimation algorithm. In both of these cases the users are not restricted in what they can select, i.e. they can select any available feature. At the next step, their current selection is modeled as a multidimensional vector (in case of an algebraic model) and compared with vectors representing predefined configurations to obtain an optimal match. Note here, that the obtained match might include features which are not selected at all or might miss certain required features, i.e. it is an approximation of what the users required. Secondly, the system might simply collect the user requirements without making a particular estimation on the best or an optimal match for the collected requirements. Rather, the system might try to produce a prototype configuration which meets the requirements. That prototype can be seen only as a starting point to obtain the proper configuration - at the next step the system developers might work on the prototype to meet the requirements in full. Thirdly, in some cases even creating a prototype will not be possible. In those cases the collected requirements might be used to configure the system manually from scratch. However, one huge advantage in this case is the fact that the requirements are formally defined as an XML document. 3.4 First Experiences with Formal Representations of Learning Scenarios We have applied the presented method for management of learning scenarios at the end of the Ephras project. Additionally, we have started a test phase for the new approach by including the wizard tool into WBT-Master. Here are in short some of the first experiences gained

67 Learning Scenarios in E-Learning Experiences from Ephras In the Ephras project the wizard tool has been presented to the teachers at the end of the project. Therefore, the teachers had already certain knowledge and feeling what can be expected from the technical realization of their ideas. Nevertheless, they could deepen their knowledge about the technical possibilities by simply using the wizard tool for exploration and reading about different QTI question types. In addition, a possibility to automatically and immediately obtain a quiz and define details of questions that have been chosen for the quiz tremendously improved the understanding of the technical possibilities of the system. This in turn, influenced the way how the teachers designed their new exercises from the didactical point of view. Basically, they focused more on such learning scenarios which are technically feasible and they also introduced new learning scenarios which can be characterized as specific learning scenarios for a computer-aided instruction, i.e., the teachers did not use such learning scenarios in a traditional classroom settings of for paper exercises because they lacked the interactive possibilities offered by an E-Learning system. On the other hand, the system developers got a far better understanding of the didactical requirements within the project. By analyzing automatically obtained quizzes, question types, and questions the system developers could identify which question types are the most important for the teachers and, more importantly, the system developers could identify the combinations of the questions which are typically used. For example, one combination that has been very frequently used was a combination of a larger text where a number of sentences might be selected with a single correct selection. In addition, there is a multiple-choice question where a single answer can be selected to replace the selected sentence from the text. The replacement itself is achieved by means of dragging-and-dropping the selected answer from the multiple-choice question. Technically, this is a very complex combination, that includes a hot-spot area question for selecting a sentence form the text (using hot-spot in this particular case is actually a workaround since QTI does not define a question type select a text ), a multiplechoice question, and a drag-and-drop question. Note that hot-spot area and dragand-drop questions include also a spatial component, i.e., the developer needs to define positions on the screen where the answers are located or might be dropped with the mouse. To implement and configure such combinations the developers introduced socalled templates, i.e. generic implementations of the desired behaviour that can be used to create instances by configuring it. The configuration process typically involves setting of a couple of parameters. However, the problem was how to recognize that the teacher wants to have a particular template. The solution for the problem is very simple. The developers simply included the created templates into the domain model for QTI as special classes that combined a number of questions into a single entity. In this way the teachers could simply explore a template as

68 Learning Scenarios in E-Learning any other class from the model and see its components. If the template fits into their scenario requirements the teachers simply include it in the scenario. Note how the meta-model through its extensibility (supported in this case by means of composition mechanism) greatly facilitated this problem. The last difficulty that the system developers have been confronted with is not directly related to the presented approach but to the QTI standard and the way how presentation of questions is defined in QTI. Basically, QTI XML files mix together the content and its presentation. For each question apart from its actual content the developer needs to define its presentation, e.g. the coordinates on the screen where the question will be presented. This is a serious design flaw since it violates a well-known principle in software development called separation of concerns [Dijkstra, 1982]. Consequently, whenever the developer modifies the content of a question its presentation has to be modified as well because the spatial relations within question need to be updated. Especially, creating template instances can be very tedious because of this difficulty. Referring to the example from above suppose that the text (from which sentences might be selected) is modified. This means that the developer needs to update the positions of hot-spot areas as well as position of areas where replacement sentences might be dropped. The only way to achieve this when using QTI is to update the template instance manually either by using a QTI editor or by editing QTI XML files with an XML editor Experiences with WBT-Master WBT-Master belongs to the class of systems that can not be fully dynamically configured, i.e. the system offers about 20 predefined configurations from which a single configuration might be selected. Additionally, the system developers can introduce a new configuration to fit the requirements of a particular learning scenario by manually creating a new training course. To select a particular configuration, the users (teachers of Civil Engineering courses at the University of Technology Graz) operated the wizard tool to explore the WBT-Master domain model and select the features which fit into their learning scenario. The system then offered an optimal preconfigured training course that was the best fit for the selected didactical features. Here two configuration strategies have been applied. The first strategy was based on a Boolean model, i.e. whenever the users select a feature all other features not related with the selected one in any of the existing configurations can not be selected anymore. However, the users can still explore all of the existing features and learn in that way more about the system. This approach has been successful with novel users of the system because at the end of a learning scenario capturing session the training course that was configured by the wizard tool was an exact match of what the users selected

69 Learning Scenarios in E-Learning For more experienced users who were already familiar with WBT-Master this approach was too restrictive. Therefore, for experienced users we based the configuration strategy on an algebraic model (a vector space model). Here, the users could select any feature to include it in their learning scenarios. The selected features were then modeled as a multidimensional vector and then compared with predefined configurations (also modeled as multidimensional vectors) to find an optimal match. The success of this configuration strategy depends strongly on the underlying vector space model. In particular, it depends on the dimensions of the vector space, as well on how these dimensions are weighted in the model. For WBT-Master we included for example communication, tutoring, self-initiated learning, testing, content creation, or student content creation as dimensions in the vector space model. At the next step each feature has been modeled as a vector in this vector space, and a particular learning scenario or a predefined configuration is simply a sum of vectors of all selected features. Obviously, by selecting appropriate dimension weights it is possible to emphasize that a particular dimension is more important than the other one. In case of WBT-Master the communication dimension and student content creation dimension have been weighted the most. The approach based on non-restrictive selection of didactical features has another important property. Basically, we have used this approach to collect the data about desired and needed learning scenarios. At the next step we have analyzed that data and tried to identify certain usage patterns of didactical features, e.g. how didactical features are typically combined. This information has been used to develop a number of new training courses that reflected the identified patterns. These training courses have been included in WBT-Master, as well as in its domain model for the wizard tool. Thus, the system evolution has been facilitated in this way. 3.5 Conclusions and Further Work This chapter argued that modern E-Learning systems which support didactical aspects of E-Learning are inevitably technically complex. Also, there is a gap between the didactical and technical aspects in such systems. Typically, this situation leads to suboptimal usage of the systems. To bridge that gap a huge effort on both sides, i.e. the teacher side as well as the system developer side is needed. In particular, in order to support didactically sophisticated learning scenarios an iterative system configuration and management process is needed. The chapter presented a possible solution for this problem based on formal representation of learning scenarios by modeling the didactical as well as technical domain in question. The first results with the presented solution were positive, especially in such E-Learning systems where a complete dynamic configuration is possible. In system where that is not the case the presented

70 Learning Scenarios in E-Learning solution tries to estimate an optimal configuration. Additionally, it supports the system evolution through the analysis of the users needs. Although the results of the presented approach are encouraging we see it only as a first step in automatic management of learning scenarios. Currently, this approach does not take into the account the dynamics of a learning scenario. Thus, the users select the didactical features which they would like to include in a learning scenario but they can not impose a structure on top of these features. For example, for collaborative writing scenario a user would include activities such as writing, reading, reflecting, commenting, and discussion. The system would offer a particular configuration that includes tools to support each of these activities. All of these tools would be presented to the students in a single user-interface and will be present at all times. However, there is a certain temporal structure which can be imposed on the top of these activities, i.e. firstly, the students need to read a particular document; secondly, the students need to write their own document; thirdly, the teacher provides comments; lastly, students are involved in reflecting activity. In parallel, the discussion activity is carried out. Thus, there is a certain (process-oriented) execution sequence within this particular learning scenario and that execution sequence should be also supported by the system. For example, during the writing activity the students should have at their disposal only a text editor to write their contributions and a discussion forum to discuss all current issues with the teacher and their peers. At the next step during the commenting activity the students can only read the teacher s comments and do not have the text editor at their disposal anymore. In this way the learning process within a learning scenario might be facilitated. Thus, the future work would include an extension of the presented approach to include management of the processoriented aspects of learning scenarios

71 4 Technology-Supported Management of Collaborative Learning Processes This chapter is an extended version of a paper published in the International Journal of Learning and Change [Helic 2006a]. This chapter deals with collaborative learning processes in a technology-enhanced learning environment and claims that a fully-fledged technological support for management of such processes is still missing. To back up that claim the chapter introduces an analysis framework for the evaluation of the state-of-the-art in technology-supported management of learning processes. This evaluation clearly reveals deficiencies in such a support and points out possible approaches for resolving the identified problems and drawbacks. As one of such approaches the chapter discusses possibilities of using the Business Process Management technology for the management of collaborative learning processes. 4.1 Introduction Although the process-oriented nature of collaborative learning in traditional settings is indisputable, both E-Learning in general and collaborative E-Learning in particular commonly neglect this fact. Typically, E-Learning adopts one of the following learning modeling approaches: the content-oriented, the tool-oriented, or the task-oriented approach [Marjanovic, 2005]. Commonly, these approaches are discrete and independent and they are rarely used simultaneously in E- learning: Content-oriented approach is mainly concerned with management of learning content in E-Learning systems by supporting authoring, structuring, delivering, sharing, re-using, and querying of the content. Additionally, an extensive tool support for the students in their daily work with the learning content is commonly provided. For example, this support includes collaborative tools for enriching the learning content by writing comments and annotations, tools for tracking the student progress with the content, or tools for adapting the content to the students preferences [Barron, 1998; Collis and Strijker, 2003; Barker, 2004]

72 Technology-Supported Management of Collaborative Learning Processes Tool-oriented approach is based on the underlying technological infrastructure. Learning sessions which follow this approach are organised around the use of a particular collaborative tool and thus only reflect the technology. For example, moderated discussions or online conferencing sessions are typical uses of a discussion forum or a chat tool, respectively [Mioduser et al., 2000]. Task-oriented approach deals with learning tasks or learning activities which the students need to perform in their learning sessions. Those tasks are typically structured in very simple learning sequences that the students need to pass in a sequential mode. Commonly, learning tasks include reading, writing, or commenting tasks and are typically associated with a specific learning content [Agostinho et al., 2002; Oliver and Herrington, 2003; Collis and Margaryan, 2004]. The main goals of this chapter are first to show that none of these approaches deals with the learning process itself, but addresses only certain parts of such a process; second, that this situation can be seen as a major drawback in E- Learning; and third, to discuss possible solutions and future developments that might provide remedy for these problems. To achieve these goals the chapter focuses on an analytical survey of the current situation, trends, standardization efforts, and systems for technology-supported management of collaborative learning processes. The data for this study was collected over the last 5 years in a number of European-wide research, commercial, and university E-Learning projects [Pfahl et al., 2004; Helic et al., 2004; Helic et al., 2005]. For the purposes of this analysis the chapter introduces a generic technologyaware analysis framework for collaborative learning processes. In the next step, the state-of-the-art in the field is compared with this framework to infer its weaknesses, drawbacks, problems, and possible strengths. Finally, the results of this analysis are used to propose remedies to the identified problems and drawbacks. 4.2 Technology-Aware Framework for Collaborative Learning Processes A significant work has been done on learning processes [McIlrath and Huitt, 1995] in general and collaborative learning processes [Reinmann-Rothmeier and Mandl, 1999; Resnick, 1989] in particular. Summarizing, collaborative learning processes are such learning processes where learning tasks are based on real-life tasks or authentic situations and typically require and motivate the co-operation or collaboration (co-construction and exchange of knowledge) of learners in a group. In addition, collaborative learning processes realize central features of a learning community, i.e. they promote the development of both individual and socially

73 Technology-Supported Management of Collaborative Learning Processes shared knowledge; support and instruct the learning group on how to reflect their individual and collective experiences, identify their learning needs, and continually evaluate their knowledge and experience development (promotion of meta-cognitive processes); initiate the sharing and negotiation of knowledge by developing of a positive learning culture; take care that the group members are structurally interrelated and remain open-minded to external knowledge resources; and strive to support the development of a group-oriented identity. In technology-enhanced learning field there are number of initiatives to develop various tools for collaborative learning [Althoff et al., 2002]. For example, one of such projects was the project called CORONET (Corporate Software Engineering Knowledge Networks for Improved Training of the Work Force) that was funded by the European Commission (IST ). The main purpose of the project was to analyze, implement and evaluate a number of tools for support of collaborative knowledge transfer processes. Each of such tools utilized the current and advanced Web technology to facilitate and speed the flow of knowledge from people possessing that knowledge to people who need to acquire it by following a particular collaborative didactical approach in a process-oriented manner. Thus, processes such as Web-based tutoring, Web-based knowledge mining, Web-based collaborative writing, and collaborative project-oriented learning have been supported. The evaluation of the project results in respect to the increase of learning effectiveness by knowledge sharing and collaborative learning generally indicated improved learning effectiveness [Pfahl et al., 2004; Helic et al., 2004; Helic et al., 2005]. The previous research and development suggests that the collaborative learning processes in an E-Learning environment contained the following five components: learning content, learning procedure, communication and collaboration facilities, technological infrastructure, and run-time execution procedure (see Figure 4.1): Learning content in technology-enhanced collaborative learning comes in various electronic formats. The formats include courseware structured according to the latest E-Learning standards such as Sharable Content Object Reference Model (SCORM), text and media-enriched documents in different formats [Baudry et al., 2005], external Web documents including discussion board contributions, various blogs [Avram et al., 2004], or wiki-like contributions [Fuchs-Kittowski et al., 2004]. Sometimes the content is not directly available in electronic form and needs first to be converted into that form. For example, a lot of learning content is still available only in text books, or is simply a part of knowledge that different participants in the learning process possess. Before such learning content might become a direct component of the learning process it needs to be extracted from the people and transformed into electronic form [Oliveira and de Souza, 2004]

74 Technology-Supported Management of Collaborative Learning Processes This, in turn, is typically achieved through communication and collaboration with other participants in the learning process [Maybury, 2002]. Figure 4.1: Generic Learning Processes Framework Learning procedure is a structured rule-based sequence of learning activities or learning tasks; those need to be executed by different participants in the learning process in order to achieve a particular learning goal [Sampson and Karampiperis, 2006]. The rules which govern the execution of such a learning sequence typically follow a particular didactic approach [Schroeder and Spannagel, 2006]. For example, for the above-mentioned collaborative project-oriented learning process applied to teaching software development the following learning procedure can be defined. First, the students need to read about different software development methods. Second, the students develop collaboratively a particular software system by following an iterative software development method including analysis, design, implementation and test phases. After each of these activities the teacher

75 Technology-Supported Management of Collaborative Learning Processes provides feedback for the students. One of the sequencing rules requires the students to repeat a particular step if the teacher s feedback is negative [Helic et al., 2005]. Communication and collaboration between participants is one of the most important aspects of the learning process. Nowadays, learning is essentially a social process where the possibility of establishing a contact with other people, discussing, exchanging, and brainstorming ideas with the peers, or learning and working together with others is of primary importance [Repman et al., 2004; Hawkes, 2001; Rico, 2003; George, 2004]. Huge success stories of so-called social software applications, such as blog applications, shared bookmarks, or wiki-based encyclopedias are typical examples of the importance of the role which communication and collaboration play in technology-enhanced learning today. Technological infrastructure and tools support different aspects of the learning process. This infrastructure includes but is not limited to user management tools, access right management tools, content management systems, content presentation tools, synchronous and asynchronous communication tools, as well as collaboration tools such as version control systems, shared content structuring and management systems, or annotation tools [Shih, 2002]. Run-time execution procedure is obtained by mapping the learning procedure onto the available tools and the technological infrastructure. Such an execution procedure typically defines composition, communication, and orchestration rules which are needed to successfully integrate and control the tools that are required to support high-level learning activities and the rules governing them [Junzhou et al., 2006; Torres et al., 2005; Lin et al., 2001]. For example, for the above mentioned project-oriented learning process [Helic et al., 2005] the execution procedure would include tools such as user and access right management system (for controlling who can do what), content viewer (for the reading of the learning content), discussion forum (for a general discussion of all issues related to the learning process), upload tool (for the uploading of the course results), feedback, annotation and tool (for providing feedback and writing comments on the students work). In addition, a number of execution rules should be defined inn order to control the way how different tools communicate and work together. For example, one rule might state that a student needs to be automatically notified by whenever the teacher provides feedback for that student; another rule provides automatic tool support for the student to repeat certain steps in the learning process if that feedback was negative (see Figure 4.2)

76 Technology-Supported Management of Collaborative Learning Processes Figure 4.2: Learning Process for Project-Oriented Learning The most significant deficiencies of the E-Learning modeling approaches mentioned above, i.e., the content, task, and tool-oriented approach when compared with this framework can be summarized as follows: Lack of possibility to formally define a particular learning goal, which the students need to achieve, as well as a lack of possibility to automatically check the students success in achieving that goal. Currently, the learning goal can be defined informally, e.g. by describing it within the learning content or within a discussion forum contribution. Obviously, checking of the students success can only be accomplished by the users. Lack of possibility to define the learning procedure, i.e., the set of learning activities structured by means of certain pedagogical rules that lead

77 Technology-Supported Management of Collaborative Learning Processes students to achieving the learning goal [Marjanovic, 2005]. For example, these learning procedures might reflect such sophisticated pedagogical approaches as problem-based learning, collaborative writing, or projectbased learning. It is important to note here the difference between such learning procedures and simple sequences of learning activities from the task-oriented approach. Lack of possibility to automatically map the learning procedure onto the available tools and the underlying technological infrastructure. The final issue which the developed analysis framework takes into account is the fact that collaborative learning processes exhibit a very dynamic nature in practice [Pfahl et al., 2004; Helic et al., 2004]. First of all, there exist a wide range of external factors which influence the way of how learning processes and their components are developed or executed a small change in these external factors leads to changes in the learning process. Also, learning processes are typically repeated by new students and therefore they are typically closely observed for their further modifications and improvements. Table 4.1 summarizes some of the typical changes in external factors and how these changes affect learning processes. External Factors Changes in knowledge and skill level Changes in organisation structure (e.g. when a team member leaves a team) Improvements in didactical approach (e.g. when a learning process is repeated with new students) New external learning content appears (e.g. through a blog or a wiki system) Changes in technological infrastructure (e.g. a new software releases) Influence on Learning Process Components Learning content needs to be adapted Learning and run-time execution procedures need to be customised Communication and collaboration practices are changed Run-time execution procedure needs to be adjusted Increased communication to explain the new approach Communication and collaboration practices might change Learning and run-time execution procedures might require customisation Run-time execution procedure needs to be adapted Table 4.1: Dynamics of Collaborative Learning Processes 4.3 State-of-the-art in Management of Collaborative Learning Processes In the university settings teachers commonly design, develop and publish their learning content using a standard E-Learning system such as Blackboard or WebCT. However, these modern E-Learning systems do not deal with learning

78 Technology-Supported Management of Collaborative Learning Processes processes per se, that is those systems support the learning process only partially. Typically, it is not possible to define the learning procedure or the pedagogical rules that will govern the learning process. Rather the teachers need to monitor the students and impose the pedagogical rules manually in order to lead the students to a particular learning goal. Apart from increased teacher workload this situation has another drawback. Basically, it is very difficult to improve the learning process for subsequent executions since the only modifications that can be made by the teachers are at the level of the learning content and not at the level of the learning procedure. This means that a subsequent execution of the same process requires that the teachers repeat their monitoring work since the system does not offer a possibility for modeling and improving of the learning procedure. Recently, some standardization, research, and development efforts have been undertaken that take into account process-oriented nature of technology-enhanced learning. Among these efforts there are two standardization initiatives - SCORM Sequencing and Navigation and IMS Learning Design, as well as an innovative E- Learning system - Learning Activity Management System. SCORM Sequencing and Navigation emerged from the IMS Simple Sequencing specification. This specification defines a standardized way of sequencing the learning content and learning activities for a particular student. Basically, it provides means for specifying so-called learning paths, which can branch according to the current learning situation. However, learning activities that occur along such a personalized and adaptive learning path are typically only reading or discussion activities. Essentially, SCORM Sequencing and Navigation might be seen only as an improved way of structuring the learning content to match the current learning situation, i.e. it can be hardly comprehended as a specification that supports management of collaborative learning processes [Kwang-Hoon et al., 2005; Yang et al., 2004]. IMS Learning Design (IMS-LD) provides a formalized way of describing activitybased learning scenarios and expressing different pedagogical concepts in the form of so-called Units-of-Learning. Units-of-Learning can be exchanged between different E-Learning systems for execution. For example, the above mentioned project-oriented learning process might be developed as an IMS-LD Unit-of-Learning that includes and structures all the learning activities such as reading, testing, analyzing, designing, and implementing. Moreover, IMS-LD defines a number of formal constructs which might be used to apply certain pedagogical rules on the top of the learning activities. Thus, IMS-LD might be seen as a formal language to specify learning procedures. Additionally, IMS-LD Units-of-Learning might refer to the external learning content (e.g., by linking the content via Web addresses) or might refer to the tools and services available in an E-Learning system [Koper and Burgos, 2005]. However, in the current version of IMS-LD those tools and services are restricted only to four simple services such

79 Technology-Supported Management of Collaborative Learning Processes as and discussion forum. Also, IMS-LD totally lacks possibilities to map the learning procedure onto the execution procedure, or to model the execution procedure. This means that this part of the learning process is basically unsupported by IMS-LD and it is up to the implementers of the IMS-LD specification how such a mapping and modeling can be accomplished. This represents a serious drawback of IMS-LD [Leo et al., 2004, Torres et al., 2005]. Learning Activity Management System (LAMS) is an E-Learning system implemented around a concept that E-Learning is as an activity-centric and workflow-based undertaking [Dalziel, 2003]. The LAMS system was strongly inspired by the IMS-LD approach of modeling learning procedures in the form of Units-of-Learning. Later on, the implementation concentrated more on management of reusable learning activities, and the core functionality of E- Learning systems such as learning content management. LAMS proves to be very successful for supporting collaborative, experiential, self-initiated, and activityoriented learning [Voerman and Phillip, 2005]. Currently, LAMS is a fullyfledged E-Learning system and its learning process management functionality cannot be used without the rest of the system. Recently, LAMS has been extended to make it possible to integrate LAMS and its learning process modules into other external E-Learning systems and tools. Nevertheless, this integration support is provided only at the level of user authentication facilities, i.e. a single-sign-on procedure is provided which authenticates a user in both LAMS and an external E-Learning system. There are no possibilities whatsoever to sequence, structure, control, or orchestrate the external system by executing learning activities in LAMS. Basically, this means that to make use of management of learning processes by LAMS, the users need to abandon their E-Learning systems and move over to LAMS, which is typically not possible. Thus, this short overview of the state-of-the-art in technology-supported management of learning processes shows that a fully-fledged support is still missing. 4.4 Supporting Learning Processes through Business Process Management To successfully cope with the complexity of the collaborative learning process and its dynamics one of the most important requirements for the next generation of technology-enhanced learning is to facilitate technology-supported management of the collaborative learning process in its entirety. Such a technological support must include tools that provide fully-fledged support for all components of the process, as well as possibilities to smoothly manage changes in that process. In particular, such a support must include and integrate: learning content management component based on the recent E-Learning standards and principles; the basic and advanced E-Learning and collaborative tools; a modeling and authoring component for the creation of sophisticated collaborative learning procedures; an automatic or semi-automatic mechanism for mapping learning

80 Technology-Supported Management of Collaborative Learning Processes procedures onto the underlying technological infrastructure (i.e., onto run-time execution procedures); components for monitoring and analyzing the students and their progress within a particular learning process; and finally facilities for altering, updating, and improving processes to be able to cope with their dynamics. One possible approach to providing such a support might be a reuse of the experiences, practices and principles from technology-supported management and automation of business processes in organizations. There are close connections between collaborative learning processes and business processes at a number of levels. For example, at the communication and collaboration level human participants in both processes try to achieve a certain goal (i.e., a business or a learning goal) by working closely together with other people in a particular social context. Further, at the procedural level, both the business process and the learning process deal with tasks and activities structured in a certain way and executed by following a set of rules, i.e. business rules or pedagogical rules (the learning procedure) respectively. Also, at the content level both of the processes work with standardized and interoperable content in electronic form the learning content or business documents. Finally, at the run-time execution and technological infrastructure level both processes rely on software systems and tools that need to be integrated, orchestrated, and synchronized by means of execution rules. Let us now look more closely at the technological solutions for the management of business processes. Recently, such solutions are typically referred to as Business Process Management (BPM) technology. BPM evolved from the workflow technology and the Web technology. Workflow is typically defined as the automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules [Hollingsworth, 1995]. The automation is achieved through a so-called workflow management system that manages and executes workflows represented in a machine-understandable way. Participants of a workflow can be either humans or software systems that execute tasks defined in the workflow in order to achieve a particular business goal. Thereby, the humans work with the so-called worklist (i.e., a list of tasks to be executed) to achieve their goal. Currently, architectural principles such as Service-Oriented Architecture (SOA) and its particular implementation in the form of Web services are adopted by the workflow technology to achieve integration between Webbased software systems operating across the organizational boundaries. Additionally, the current workflow technology is based extensively on Web standards such as Extensible Markup Language (XML) to facilitate document and data exchange, to create definitions of service interfaces or to manage interoperable representations of workflows [Hollingsworth, 2004]. For example, today workflows are typically defined using an XML-based language called

81 Technology-Supported Management of Collaborative Learning Processes Business Processes Execution Language (BPEL). Obviously, basing management of collaborative learning processes on the BPM technology will bootstrap its development. Nevertheless, there are a few very important research and development steps which need to be realized for a successful application of BPM in learning process management. Those development steps take a top-down approach to supporting the whole management cycle for learning processes, i.e., first, facilities for creating and modeling of collaborative learning processes are addressed; second, mechanisms for automatic mapping, deploying, and executing of learning processes within the available technological infrastructure are taken into account; and finally, integration with existing user-interfaces are considered. Step 1: Authoring and modeling of collaborative learning processes should be supported. Business processes are typically defined using a graphical notation, such as Business Process Modeling Notation (BPMN) or Unified Modeling Language (UML). To simplify authoring of learning processes a similar graphical notation should be taken as the authoring basis. For example, a learning process authoring tool might support a graphical notation for defining learning processes including learning tasks, activities, procedural pedagogical rules, participants and their roles. Since notations such as BPMN are meta-modeling notations, i.e. such notations introduce only abstract modeling elements such as task, activity, participant, role or rule; a pedagogy-specific vocabulary based on these abstract elements should be constructed. For example, learning tasks such as reading, writing, testing or reflecting might be introduced. Similarly, participant roles such as teacher, tutor, or student should also become a part of this pedagogical vocabulary. Step 2: Mechanism for the mapping of conceptual learning process onto their executable counterparts should be introduced. Such a mapping must be possible with minimal user intervention, automatically or semi-automatically. Allowedly, this is a very difficult task, but a reasonable remedy can be found in templatebased authoring, where the teacher specifies only a couple of parameters for a common template, for which an executable model (e.g. in BPEL) already exists. Step 3: Seamless integration of run-time execution procedures and a general E- Learning system must be achieved. For example, E-Learning functionality might be exposed as a collection of interoperable and through open standards accessible functionality, e.g. in the form of Web services. Prior to that, a specification of interfaces for these Web services should be made. For example, suppose that a run-time learning process requires upload functionality to store student contributions. Typically, the E-Learning system offers upload functionality by means of a graphical user interface that communicates with a Web server. However, to integrate it with run-time learning processes upload functionality must be made available through a Web service with a clearly specified interface

82 Technology-Supported Management of Collaborative Learning Processes Typically, the created Web service will operate only as a mediator between the specified interface and the concrete interface of the E-Learning system. Step 4: Seamless integration of worklist user interface with the existing user interface of an E-Learning system must be achieved. For example, user interface of the worklist could follow the hypertext interface paradigm, e.g. different tasks might be represented as hyperlinks. By clicking on a particular task-related link users access the associated learning environment, i.e. a system tool associated with the task and the learning content needed to accomplish the task. By following these steps the following generic architecture of a Learning Process Manager (LPM) might be inferred (see Figure 4.3). Figure 4.3: LPM Manager Architecture Learning Process Execution Engine is the main component of LPM. The execution engine is responsible for the concurrent execution of a number of learning processes, which are typically called active learning processes. Each active learning process is an instance of a learning process definition (processes are defined using BPEL specification), which is associated with a unique execution context. Such an execution context keeps all the necessary information that is needed to, for example, identify the process instance, associate it with a unique user (e.g., a unique student), exchange data between the process and the external tools, provide learning and working tasks for users, store the current process state, or make execution sequencing decisions according to the execution rules. Thus, the execution engine leads the students through the learning sequence by creating the learning tasks and sending those tasks to the worklist manager, one task at a time

83 Technology-Supported Management of Collaborative Learning Processes Learning Process Definition Module is based on the defined pedagogical vocabulary. This vocabulary defines a number of typical collaborative learning activities such as reading, discussing, writing, testing, communicating, or brainstorming, as well as a number of typical user roles such as teacher, student, or tutor. On top of this vocabulary a number of learning process templates are prepared. These templates reflect typical collaborative learning processes such as project-oriented learning or collaborative writing. Each template is defined through the learning procedure and a corresponding execution procedure. For example, collaborative writing template defines the following learning procedure. First, the students need to read a document that explains important concepts from the subject domain. Second, the students are informed about the topic of their written work. Third, the students discuss the topic. Fourth, the students write in collaboration a document about their topic. In parallel, they discuss their work and all other relevant issues. The corresponding execution procedure maps the learning activities from the learning procedure on the execution steps and connects them with the appropriate tools. For example, the discussion activity is mapped onto a discussion forum, the collaborative writing activity is mapped onto an upload tool associated with a version control system, and the whole process is connected to a user authentication system. Thus, the authoring process is based on selecting an appropriate template and defining a number of parameters to obtain a particular process definition. Typically, this process only involves associating of the learning activities from the learning procedure with the learning content. For example, to obtain a process definition instance for the collaborative writing template the teacher needs to supply two documents (or Web addresses of two documents). The first document is related to the first reading activity and explains the concepts from the subject domain. The second document is related to the second reading activity and describes the topic about which the students need to write their documents. Worklist Manager keeps track about learning and working tasks for all users and for all running processes. Also, it provides an interface for retrieving all completed and all active activities for a particular user and a particular process. The worklist manager interface might also be used to set learning activities as completed. Additionally, a call-back mechanism provides a possibility to inform the execution engine about the completion of learning and working activities. Monitoring and Analysis Module provides a graphical user interface that allows the teachers to investigate all properties of a running learning process. For example, the time needed to execute a particular learning activity, or to branch and sequence learning activities can be easily observed. External Tools, E-Learning System, and Communication and Collaboration Tools expose their functionality in form of Web services to facilitate communication and interaction with the execution engine. Obviously, the exposed functionality

84 Technology-Supported Management of Collaborative Learning Processes closely reflects the learning activities that are defined in the pedagogical vocabulary. For example, to support the test learning activity, the E-Learning system has a Web service to retrieve the test results for a particular student or to set test results for a particular student. The latter functionality is needed whenever the teacher revises a student s test, and the former functionality is needed whenever the execution engine needs the test results to make a sequencing decision. Currently, the process execution engine, the worklist manager, and the monitoring and analysis modules are available as the standard components of any BPM product. Therefore, the main research and development focus will be on development of the learning processes definition module, i.e., the pedagogical vocabulary, and the learning procedure templates that can be automatically mapped onto executable processes. Additionally, development of standardized interfaces for the E-Learning functionality and the user-interface modules should be carried out, in order to integrate both of them into the BPM process execution engine and the worklist manager, respectively. 4.5 Sample Learning Process in LPM Let us introduce here a sample learning process to practically illustrate the learning process lifecycle and behavior of a learning process in real-world settings. Also, the sample learning process will be used throughout the rest of the chapter to explain the implementation aspects of the LPM system. Suppose that we have a university course dealing with the basics of computer operating systems, in particular with the basics of the Linux operating system. There are 10 students who would like to participate in that course. The course is held by a university teacher who is currently on a research trip abroad. Therefore, the teacher would like to carry out the course in an online mode. However, the teacher does not want to have only a simple Web-based reading course where the students simply read a number of documents prepared by the teacher. Rather, the teacher would like to involve the students in a learning process where they participate in discussions with their peers, collaborate with each other, and take part in online tests. More precisely, the teacher imagines the following learning procedure. First, the students are supposed to read through some learning content. The learning content includes a remote PDF document and a Web site on the basics of Linux, as well as a learning course prepared with an E-Learning system. Second, the students are free to write comments, ask questions, or brainstorm certain ideas from the learning course using the annotation feature which is available in the system. Third, the students are supposed to use a discussion forum provided by the E- Learning system to discuss a number of topics from the subject matter. The topics are predefined by the teacher. Fourth, the students need to take an online test

85 Technology-Supported Management of Collaborative Learning Processes using the test tool from the E-Learning system. Lastly, all students that achieve more than 50% of correct answers in the test finish the course successfully. All other students need to read an additional document provided by the teacher and then repeat the test. If the students fail the test for the second time they fail the course, otherwise they pass the course successfully. Let us investigate now the learning process lifecycle and run-time properties of this simple learning process. In the first phase of the process (modeling phase) the teacher prepares the learning content by reusing some online material and creating the learning course using the E-Learning system. Subsequently, the teacher creates the learning procedure by using a process definition tool offered by the LPM system. Lastly, the teacher maps the learning procedure onto the execution procedure by operating a process mapping tool from the LPM system. Here, the teacher instructs the LPM system which E-Learning system and which tools from that E-Learning system are used. In the second phase (learning or execution phase) the students authenticate with the LPM system and choose the sample learning process. The system runs the execution procedure and presents learning tasks to the students which they need to accomplish. Since the learning procedure created by the teacher is a simple sequential procedure the students get one learning task at a time, i.e. a couple of reading learning tasks, discussion learning task, test learning task, and then if needed another reading learning task and finally again test learning task. Whenever a student finishes a learning task the LPM system is notified and the student gets the next learning task from the sequence. At any time the students can monitor their progress within the learning process, i.e. they always can see which learning task they have finished (see Figure 4.4). During the learning phase, the teacher monitors the students progress using the LPM system. Additionally, the teacher discusses all course related issues with the students, answers their questions, or clarifies certain topics from the subject matter by using the E-Learning system. Further, the LPM system provides certain working tasks for the teacher as well. For example, whenever a student finishes the test the LPM system notifies the teacher. In turn, the teacher needs to revise and mark the student s test (see Figure 4.5). At the end, the teacher submits the mark for the student s test to the LPM system

86 Technology-Supported Management of Collaborative Learning Processes Figure 4.4: Student Participation in a Learning Process Figure 4.5: Teacher Participation in a Learning Process

87 Technology-Supported Management of Collaborative Learning Processes In the third phase (observation and analysis) phase, which runs in parallel with the second phase, the teacher can use monitoring tools provided by the LPM system for analyzing of the learning process (see Figure 4.6). For example, the teacher might notice that the students need a lot of time to read a particular document. The teacher comes to the conclusion that the content is not adjusted to the knowledge level of the students and improves this situation. Figure 4.6: Analyzing a Learning Process 4.6 Implementation Aspects of LPM Learning Process Execution Engine The current implementation of LPM is based on an Open Source implementation of BPEL specification called ActiveBPEL [ActiveBPEL 2006]. The engine is capable of executing processes defined in BPEL and provides monitoring and analyzing tools. In addition, the commercial version of the engine provides a graphical process definition tool. This tool works directly with BPEL processes. Generally, BPEL processes are simply Web services that orchestrate other Web services and define rules for such an orchestration. Thus, ActiveBPEL provides a

88 Technology-Supported Management of Collaborative Learning Processes Web service interface to each BPEL process that it executes. This interface is used to start a particular process, and after the process has been successfully started the engine follows execution and orchestration rules and communicated with external Web services. This communication goes in both directions, i.e. outgoing and incoming directions. In the case of outgoing communication the engine contacts an external Web service, invokes a particular functionality of that service and passes all the necessary parameters. On the other hand, in the case of incoming communication the engine stops the execution of the process and waits until an external service contacts the process. Depending on the parameters that are submitted and the rules defined by the process the engine decides hot to proceed with the execution of the process. The main purpose of a BPEL process is to coordinate and orchestrate a number of Web services and in this way achieve a desired functionality. BPEL processes work with SOAP-based Web services Worklist Manager Generally, a BPEL process does not take into account human users of the system but works solely with distributed functionality provided by different Web services. However, in LPM human users of the system work with worklist manager that keeps track of tasks and activities that human users need to perform. In the current implementation of LPM worklist manager is implemented as a Web service that communicates with the BPEL engine and BPEL processes that are executed by the engine. Worklist manager service provides possibilities to add activities (i.e. work items), pass parameters for these activities (e.g. user roles, external learning resources, and similar), and uniquely identify users that need to consume these activities. Listing 4.1 shows the WSDL interface definition of the worklist manager service. <?xml version="1.0" encoding="utf-8"?> <wsdl:definitions targetnamespace=" xmlns:apachesoap=" xmlns:impl=" xmlns:intf=" xmlns:soapenc=" xmlns:wsdl=" xmlns:wsdlsoap=" xmlns:xsd=" <wsdl:types> <schema targetnamespace=" xmlns=" <import namespace=" <complextype name="arrayof_xsd_string">

89 Technology-Supported Management of Collaborative Learning Processes <complexcontent> <restriction base="soapenc:array"> <attribute ref="soapenc:arraytype" wsdl:arraytype="soapenc:string[]"/> </restriction> </complexcontent> </complextype> <complextype name="workitemproperty"> <sequence> <element name="name" nillable="true" type="xsd:string"/> <element name="value" nillable="true" type="xsd:anytype"/> </sequence> </complextype> <complextype name="arrayofworkitemproperty"> <complexcontent> <restriction base="soapenc:array"> <attribute ref="soapenc:arraytype" wsdl:arraytype="impl:workitemproperty[]"/> </restriction> </complexcontent> </complextype> <complextype name="workitem"> <sequence> <element name="properties" nillable="true" type="impl:arrayofworkitemproperty"/> <element name="role" nillable="true" type="xsd:string"/> <element name="type" nillable="true" type="xsd:string"/> </sequence> </complextype> <complextype name="arrayofworkitem"> <complexcontent> <restriction base="soapenc:array"> <attribute ref="soapenc:arraytype" wsdl:arraytype="impl:workitem[]"/> </restriction> </complexcontent> </complextype> </schema> </wsdl:types> <wsdl:message name="consumeworkitemsrequest"> <wsdl:part name="in0" type="xsd:long"/> </wsdl:message> <wsdl:message name="addworkitemresponse"> </wsdl:message> <wsdl:message name="addworkitemrequest"> <wsdl:part name="in0" type="xsd:long"/> <wsdl:part name="in1" type="impl:arrayof_xsd_string"/> </wsdl:message> <wsdl:message name="consumeworkitemsresponse">

90 Technology-Supported Management of Collaborative Learning Processes <wsdl:part name="consumeworkitemsreturn" type="impl:arrayofworkitem"/> </wsdl:message> <wsdl:porttype name="worklistmanagerservice"> <wsdl:operation name="addworkitem" parameterorder="in0 in1"> <wsdl:input message="impl:addworkitemrequest" name="addworkitemrequest"/> <wsdl:output message="impl:addworkitemresponse" name="addworkitemresponse"/> </wsdl:operation> <wsdl:operation name="consumeworkitems" parameterorder="in0"> <wsdl:input message="impl:consumeworkitemsrequest" name="consumeworkitemsrequest"/> <wsdl:output message="impl:consumeworkitemsresponse" name="consumeworkitemsresponse"/> </wsdl:operation> </wsdl:porttype> <wsdl:binding name="worklistmanagerservicesoapbinding" type="impl:worklistmanagerservice"> <wsdlsoap:binding style="rpc" transport=" <wsdl:operation name="addworkitem"> <wsdlsoap:operation soapaction=""/> <wsdl:input name="addworkitemrequest"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:input> <wsdl:output name="addworkitemresponse"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:output> </wsdl:operation> <wsdl:operation name="consumeworkitems"> <wsdlsoap:operation soapaction=""/> <wsdl:input name="consumeworkitemsrequest"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:input> <wsdl:output name="consumeworkitemsresponse"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:output>

91 Technology-Supported Management of Collaborative Learning Processes </wsdl:operation> </wsdl:binding> <wsdl:service name="worklistmanagerserviceservice"> <wsdl:port binding="impl:worklistmanagerservicesoapbinding" name="worklistmanagerservice"> <wsdlsoap:address location=" </wsdl:port> </wsdl:service> </wsdl:definitions> Listing 4.1 WSDL of Worklist Manager Service This functionality is invoked from a particular BPEL process to set the current work items for a particular user. Listing 4.2 shows an excerpt from a BPEL process that adds a new Testing work item for a student. <!-- add a question test activity to the worklist --> <assign> <copy> <from part="startreturn" variable="started"/> <to part="in0" variable="wi"/> </copy> <copy> <from> <in1 soapenc:arraytype="soapenc:string[7]" xmlns="" xmlns:soapenc=" xmlns:xsi=" xsi:type="soapenc:array"> <item xsi:type="soapenc:string">questiontest</item> <item xsi:type="soapenc:string">student</item> <item xsi:type="soapenc:string">url= ster/courses/linux_quiz_room.htm</item> <item xsi:type="soapenc:string">title=test on the basics of Linux</item> <item xsi:type="soapenc:string">message=(please note: When you finish the test you will need to wait for the teacher to mark it!)</item> <item xsi:type="soapenc:string">observable=test_1_result</item> <item xsi:type="soapenc:string">completed_message=(please note: Now, you need to wait for the teacher to mark the test!)</item> </in1> </from> <to part="in1" variable="wi"/> </copy> </assign>

92 Technology-Supported Management of Collaborative Learning Processes <invoke inputvariable="wi" name="addworkitem_questiontest_1" operation="addworkitem" outputvariable="wi_response" partnerlink="wlmanagerinvoker" porttype="ns1:worklistmanagerservice"> </invoke> <!-- wait for a message from the client that the test activity has been finished --> <receive name="questiontest_1_finished_receive" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf"> <correlations> <correlation set="cs1"/> </correlations> </receive> Listing 4.2 Part of a BPEL process that adds a Testing activity for a student On the other hand, worklist manager contacts the BPEL engine whenever a user has finished with a particular work item, thus signaling to the engine to continue with the execution of the process. Listings 4.3 and 4.4 show the SOAP messages that are exchanged between the BPEL engine and worklist manager service to add a test activity for students. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" xmlns:xsd=" xmlns:xsi=" <soapenv:body> <ns1:addworkitem soapenv:encodingstyle=" xmlns:ns1=" <in0 href="#id0"/> <in1 href="#id1"/> </ns1:addworkitem> <multiref id="id0" soapenc:root="0" soapenv:encodingstyle=" xsi:type="xsd:long" xmlns:soapenc=" <multiref id="id1" soapenc:root="0" soapenv:encodingstyle=" xsi:type="soapenc:array" xmlns="" xmlns:bpws=" xmlns:ns1=" xmlns:ns2=" xmlns:ns3=" xmlns:ns4=" xmlns:soapenc=" <item xsi:type="soapenc:string">questiontest</item>

93 Technology-Supported Management of Collaborative Learning Processes <item xsi:type="soapenc:string">student</item> <item xsi:type="soapenc:string">url= inux_quiz_room.htm</item> <item xsi:type="soapenc:string">title=test on the basics of Linux</item> <item xsi:type="soapenc:string">message=(please note: When you finish the test you will need to wait for the teacher to mark it!)</item> <item xsi:type="soapenc:string">observable=test_1_result</item> <item xsi:type="soapenc:string">completed_message=(please note: Now, you need to wait for the teacher to mark the test!)</item> </multiref> </soapenv:body> </soapenv:envelope> Listing 4.3 SOAP Message from BPEL Engine for Worklist Manager Service <soapenv:envelope xmlns:soapenv=" xmlns:xsd=" xmlns:xsi=" <soapenv:body> <ns1:addworkitemresponse soapenv:encodingstyle=" xmlns:ns1=" </soapenv:body> </soapenv:envelope> Listing 4.4 SOAP Message from Worklist Manager Service for BPEL Engine To support the interaction with users worklist manager has another bidirectional communication channel. Basically, worklist manager communicates with LPM integrated user interface adding new working items and listening to a notification from the user that a particular work item has been completed. Currently, this communication channel is implemented by means of links incorporated into the integrated user interface which users need to follow. For example, when users finish with a work item they are supposed to click on a socalled Finalize link. The corresponding HTTP request invokes a server-side functionality that passes the user and work item data to worklist manager. The manager marks then the work item as completed and subsequently contacts the BPEL engine. The engine might answer in one of the following ways: Provides a new work item. Sends a message that there are currently no new work items but the process is still running, i.e. the user need to check later if there are any new work items. Notifies the manager that the process has been completed

94 Technology-Supported Management of Collaborative Learning Processes After the manager receives the answer from the engine it sends an HTTP response to the LPM user interface and presents the engine response to the user E-Learning Services In addition to the worklist manager service the BPEL engine communicates also with Web services that expose the functionality of an E-Learning system. Currently, integration with WBT-Master has been implemented. Thus, a number of Web services have been developed offering an interface for communication with WBT-Master. All WBT-Master Web services are SOAP-based Web services that act only as simple proxies to the WBT-Master functionality. Here is a partial list of such Web services: Services for login/logout mechanism. These allow a so-called single-sign on procedure for LPM and WBT-Master and are therefore important to support the integrated user interface Service for retrieving learning resources from WBT-Master. This service is important for supporting a simple reading activity in a learning process. Service for uploading learning resources onto WBT-Master. This service is important for writing, uploading, or discussion activity. Services for retrieving tests from WBT-Master, submitting test answers by the students, retrieving test answers and marking them by the teachers, submitting and retrieving test results (see Listing 4.5 for WSDL of Get Test Results service). <?xml version="1.0" encoding="utf-8"?> <wsdl:definitions targetnamespace=" xmlns:apachesoap=" xmlns:impl=" xmlns:intf=" xmlns:soapenc=" xmlns:wsdl=" xmlns:wsdlsoap=" xmlns:xsd=" <wsdl:message name="getresultrequest"> <wsdl:part name="in0" type="xsd:string"/> <wsdl:part name="in1" type="xsd:string"/> </wsdl:message> <wsdl:message name="getresultresponse"> <wsdl:part name="getresultreturn" type="xsd:int"/> </wsdl:message> <wsdl:porttype name="gettestresultservice"> <wsdl:operation name="getresult" parameterorder="in0 in1">

95 Technology-Supported Management of Collaborative Learning Processes <wsdl:input message="impl:getresultrequest" name="getresultrequest"/> <wsdl:output message="impl:getresultresponse" name="getresultresponse"/> </wsdl:operation> </wsdl:porttype> <wsdl:binding name="gettestresultservicesoapbinding" type="impl:gettestresultservice"> <wsdlsoap:binding style="rpc" transport=" <wsdl:operation name="getresult"> <wsdlsoap:operation soapaction=""/> <wsdl:input name="getresultrequest"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:input> <wsdl:output name="getresultresponse"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:service name="gettestresultserviceservice"> <wsdl:port binding="impl:gettestresultservicesoapbinding" name="gettestresultservice"> <wsdlsoap:address location=" </wsdl:port> </wsdl:service> </wsdl:definitions> Listing 4.5 WSDL of Get Test Result WBT-Master Service Typically, whenever the worklist manager notifies the BPEL engine that a particular activity has been finished the engine communicates with certain WBT- Master services and gets in this way the data needed to continue with the execution of the process. For example, the sample learning process contains the test activity. When a student finishes the test, the teacher obtains a test marking activity. To collect the data needed for that activity (i.e. test answers) the engine communicates with the corresponding WBT-Master service. Consequently, when the teacher finishes with the reviewing of the test answers the test results are submitted to WBT-Master via a corresponding service. Lastly, before the BPEL engine decides what is the next activity for the student the engine contacts a WBT-Master service to retrieve the test results for that student (see Listing

96 Technology-Supported Management of Collaborative Learning Processes and 4.7 for SOAP messages exchanged between the BPEL engine and a corresponding WBT-Master service). <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" xmlns:xsd=" xmlns:xsi=" <soapenv:body> <ns1:getresult soapenv:encodingstyle=" xmlns:ns1=" <in0 xsi:type="xsd:string">linux_quiznsherbak </in0> <in1 xsi:type="xsd:string">dhelic</in1> </ns1:getresult> </soapenv:body> </soapenv:envelope> Listing 4.6 SOAP message from BPEL Engine to a WBT-Master service <soapenv:envelope xmlns:soapenv=" xmlns:xsd=" xmlns:xsi=" <soapenv:body> <ns1:getresultresponse soapenv:encodingstyle=" xmlns:ns1=" <getresultreturn href="#id0"/> </ns1:getresultresponse> <multiref id="id0" soapenc:root="0" soapenv:encodingstyle=" oding/" xsi:type="xsd:int" xmlns:soapenc=" >17</multiRef> </soapenv:body> </soapenv:envelope> Listing 4.7 SOAP message from a WBT-Master service to BPEL Engine Once when the engine has that information it can evaluate the rule defined in the process and decide on the next activity for the student (see Listing 4.8 for an example of such a rule where the test and some additional reading for the student is repeated until the student gets at least 11 points). <while condition="11 > bpws:getvariabledata('test_result')"> <sequence> <!-- add a reading activity to the worklist --> <assign>

97 Technology-Supported Management of Collaborative Learning Processes <copy> <from part="startreturn" variable="started"/> <to part="in0" variable="wi"/> </copy> <copy> <from> <in1 soapenc:arraytype="soapenc:string[5]" xmlns="" xmlns:soapenc=" xmlns:xsi=" xsi:type="soapenc:array"> <item xsi:type="soapenc:string">reading</item> <item xsi:type="soapenc:string">student</item> <item xsi:type="soapenc:string">url= </item> <item xsi:type="soapenc:string">title=another Linux tutorial</item> <item xsi:type="soapenc:string">message=(unfortunately, you didn't pass the test! Try to improve your knowledge by reading this document!)</item> </in1> </from> <to part="in1" variable="wi"/> </copy> </assign> <invoke inputvariable="wi" name="addworkitem_reading_4" operation="addworkitem" outputvariable="wi_response" partnerlink="wlmanagerinvoker" porttype="ns1:worklistmanagerservice"> </invoke> <!-- wait for a message from the client that the reading activity has been finished --> <receive name="reading_4_finished_receive" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf"> <correlations> <correlation set="cs1"/> </correlations> </receive> <invoke inputvariable="rf" name="reading_4_finished_invoke" operation="workitemcompleted" outputvariable="rf_response" partnerlink="cmanagerinvoker" porttype="ns3:callbackmanagerservice"> </invoke> <reply name="reading_4_finished_reply" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf_response"> </reply> <!-- add a question test activity to the worklist -->

98 Technology-Supported Management of Collaborative Learning Processes <assign> <copy> <from part="startreturn" variable="started"/> <to part="in0" variable="wi"/> </copy> <copy> <from> <in1 soapenc:arraytype="soapenc:string[7]" xmlns="" xmlns:soapenc=" xmlns:xsi=" xsi:type="soapenc:array"> <item xsi:type="soapenc:string">questiontest</item> <item xsi:type="soapenc:string">student</item> <item xsi:type="soapenc:string">url= ster/courses/linux_quiz_room.htm</item> <item xsi:type="soapenc:string">title=test on the basics of Linux</item> <item xsi:type="soapenc:string">message=(please note: When you finish the test you will need to wait for the teacher to mark it!)</item> <item xsi:type="soapenc:string">observable=test_1_result</item> <item xsi:type="soapenc:string">completed_message=(please note: Now, you need to wait for the teacher to mark the test!)</item> </in1> </from> <to part="in1" variable="wi"/> </copy> </assign> <invoke inputvariable="wi" name="addworkitem_questiontest_2" operation="addworkitem" outputvariable="wi_response" partnerlink="wlmanagerinvoker" porttype="ns1:worklistmanagerservice"> </invoke> <!-- wait for a message from the client that the test activity has been finished --> <receive name="questiontest_2_finished_receive" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf"> <correlations> <correlation set="cs1"/> </correlations> </receive> <invoke inputvariable="rf" name="questiontest_2_finished_invoke" operation="workitemcompleted" outputvariable="rf_response" partnerlink="cmanagerinvoker" porttype="ns3:callbackmanagerservice"> </invoke>

99 Technology-Supported Management of Collaborative Learning Processes <reply name="questiontest_2_finished_reply" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf_response"> </reply> <!-- add a test marking activity to the worklist --> <assign> <copy> <from part="startreturn" variable="started"/> <to part="in0" variable="wi"/> </copy> <copy> <from> <in1 soapenc:arraytype="soapenc:string[8]" xmlns="" xmlns:soapenc=" ding/" xmlns:xsi=" xsi:type="soapenc:array"> <item xsi:type="soapenc:string">testmarking</item> <item xsi:type="soapenc:string">teacher</item> <item xsi:type="soapenc:string">url= /wbtmaster/courses/linux_quiz_room.htm</item> <item xsi:type="soapenc:string">title=test on the basics of Linux</item> <item xsi:type="soapenc:string">target_endpoint_address=htt p://coronet2.iicm.edu/axis/services/gettestresultservice </item> <item xsi:type="soapenc:string">target_namespace= ronet.iicm.edu/lpm/gettestresultservice</item> <item xsi:type="soapenc:string">test_name=linux_quiznsherbak </item> <item xsi:type="soapenc:string">notify_observable=test_1_re sult</item> </in1> </from> <to part="in1" variable="wi"/> </copy> </assign> <invoke inputvariable="wi" name="addworkitem_testmarking_2" operation="addworkitem" outputvariable="wi_response" partnerlink="wlmanagerinvoker" porttype="ns1:worklistmanagerservice"> </invoke> <!-- wait for a message from the client that the test marking activity has been finished -->

100 Technology-Supported Management of Collaborative Learning Processes <receive name="testmarking_2_finished_receive" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf"> <correlations> <correlation set="cs1"/> </correlations> </receive> <invoke inputvariable="rf" name="testmarking_2_finished_invoke" operation="workitemcompleted" outputvariable="rf_response" partnerlink="cmanagerinvoker" porttype="ns3:callbackmanagerservice"> </invoke> <reply name="testmarking_2_finished_reply" operation="workitemcompleted" partnerlink="cmanagerreciever" porttype="ns3:callbackmanagerservice" variable="rf_response"> </reply> <!-- repeat depending on test results --> <assign> <copy> <from variable="rf" part="in4" query="/in4/string"/> <to variable="test_result"/> </copy> </assign> </sequence> </while> Listing 4.8 Decision rule in a BPEL process Learning Process Definition Module Obviously, BPEL specification and BPEL processes are technically very complex. Thus, it is impossible for non-technical users of the system to define such BPEL processes. Therefore a number of BPEL templates has been prepared that reflect typical collaborative learning scenarios. A simple graphical user interface for the teachers has been prepared where only parameters for single learning activities need to be defined in order to obtain an instance of a collaborative learning scenario. However, in certain cases such templates are not flexible enough and the teachers need a possibility to define a new collaborative learning process from scratch. For that purpose a simple XML application on top of the pedagogical vocabulary has been developed. A particular XML document describes a learning procedure only in terms of the pedagogical vocabulary by defining a number of learning activities and parameters pedagogically relevant to these activities (see Listing 4.9). <process name="testprocess"> <workitem type="reading"> <role>student</role>

101 Technology-Supported Management of Collaborative Learning Processes <url> <title>an introduction to E-Learning</title> </workitem> <workitem type="discussion"> <role>student</role> <url> <title>discuss the basic concept of E-Learning</title> </workitem> </process> Listing 4.9 Learning process defined with the pedagogical vocabulary A very simple graphical user interface allows teachers to create such learning scenarios (see Figures 4.7 and 4.8). Figure 4.7 Selecting a learning activity for a learning scenario To obtain a complete BPEL process such pedagogical XML-based learning procedures are transformed onto a BPEL files using XSLT technology. Moreover, to ensure the extensibility of the pedagogical vocabulary XSLT transformations can be easily extended by adding a new type of a learning activity and corresponding BPEL output

102 Technology-Supported Management of Collaborative Learning Processes Figure 4.8 Defining properties of a learning activity 4.7 Conclusion This chapter analyzed the state-of-the-art in technology-supported management of collaborative learning processes by comparing the current trends and developments in this field with a generic analysis framework for such processes. The analysis identified the problems, drawbacks, and weaknesses of such a support. The critical components that are still missing include facilities for formally modeling and defining of sophisticated learning procedures with clearly defined pedagogical rules and a learning goal, as well as an automatic or semiautomatic mechanism for the mapping of such procedures onto the available technological infrastructure of an organization. Therefore, there exists a necessity for a new generation of standards, architectures, and system implementations that will take into account these missing components. As a first step in that direction this chapter considered an application of Business Processes Management technology for the management of collaborative learning processes. The main reason for this proposal is the fact that close connections between collaborative learning processes and business processes can be established. For example, in both cases users, by closely working and learning together, try to achieve a certain business or learning goal by following a number