Teaching-Material Design Center: An ontology-based system for customizing reusable e-materials

Computers & Education 46 (2006) 458 470 www.elsevier.com/locate/compedu Teaching-Material Design Center: An ontology-based system for customizing reusable e-materials Hei-Chia Wang, Chien-Wei Hsu Institute of Information Management, National Cheng Kung University,, Ta-Sheuh Road, 70 Tainan, Taiwan Abstract Use of electronic teaching materials (e-material) to support teaching is a trend. e-material design is therefore an important issue. Currently, most e-material providers over a package of solutions for diverent purposes. However, not all teachers and learners need everything from a single package. A preferable alternative is to Wnd useful material from diverent packages and combine them for a particular course. Currently, most educators collect the material manually, which is time-consuming and may result in missed material. In this paper, we describe a system the Teaching-Material Design Center, which follows the standard of Sharable Content Object Reference Model to separate e-material for use as teaching templates and learning objects and to label the material with use of semantic metadata for searching. This system can Wnd existing teaching templates and learning objects for e-material designers and provide a convenient environment for constructing customized e-material for diverent requirements. We describe the implementation and evaluation of the proposed system for a course. Our system is eycient in Wnding teaching templates and learning objects and shortening the e-material development process. 2005 Elsevier Ltd. All rights reserved. Keywords: Authoring tools and methods; E-material; Course template; Learning object semantic metadata; SCORM * Corresponding author. Tel.: +886 6 2757575x53724; fax: +886 6 236262. E-mail addresses: hcwang@mail.ncku.edu.tw (H.-C. Wang), kerio68@ms39.hinet.net (C.-W. Hsu). 0360-35/$ - see front matter 2005 Elsevier Ltd. All rights reserved. doi:0.06/j.compedu.2005.09.005

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 459. With the Xourishing development of the Internet, mass communication, and multimedia, diverent options exist for education materials. Computer-based teaching activities attract a lot of attention because learners can learn at any time and any place alone. Through the Learning Management System (LMS), students can use e-material to learn a topic according to their own needs. Because of the prevalence of e-learning and the requirements of courseware, educational resources from diverent providers become more diverse. The development of courseware is being inxuenced by the development of information technology. Third-generation teaching material in LMS, applied in most current systems, is hypermedia courseware (Aroyo, Dicheva, & Cristea, 2002). The novelty is the lack of constraints on viewing the material in a certain order. These days, the content of teaching material is no longer limited to local access. e-material can be obtained from diverent Web sites and authors, which changes the activity of courseware authoring. These changes can benewt both producers and consumers. One of the major benewts is avoiding repeatedly designing similar materials (Bohl, Schellhase, Sengler, & Winand, 2002). Thus, can select portions of e-materials from diverent providers and assemble them, which can provide more abundant teaching content for learners. This selectivity feature sounds attractive. However, under the current conditions, this feature is not easily achieved. First, there is no uniwed standard to describe e-material. Many organizations have proposed metadata to describe digital learning resources to deal with inconsistent formats. Among them is the Sharable Content Object Reference Model (SCORM), dewned by the US Department of National Defense (Advanced Distributed Learning, 200). SCORM allows for a standardized Xow to Wnd, retrieve and reuse useful resources (Silva, Lucena, & Fuks, 200). Besides the standardization issue, course sequencing has become another important research issue (Fischer, 200). A noteworthy question is how to provide an environment for Wnding and integrating suitable e-material (Paulsson & Naeve, 2003). Our proposal for reusing e-material from diverent providers and integrating them for a particular course involves dividing e-material into teaching templates and learning objects. We provide a suggested workxow for reusing the material and designing a packet. Semantic metadata, following the SCORM Content Aggregation Model, had also been used to describe e-material. Two ontologies, course ontology and content ontology, are designed to support retrieval. Our system, the Teaching-Material Design Center (TMDC), not only supports teaching template reuse and design but also integrates and sequences learning objects. We discuss the implementation and evaluation of TMDC for a proposed course. Our system can help the courseware author search for design and adapt e-material using a recommended workxow. 2. Background 2.. Learning objects Teaching materials are developed according to the needs of teaching. Currently, many studies on authoring teaching material focus on designing learning objects. In this paper, we use the term learning object for lack of unanimous dewnition (Paulsson & Naeve, 2003; Spalter & Dam, 2003).

460 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 A learning object in the educational Weld is considered as any kind of material that can be reused in teaching, such as a lesson plan, video, or section of a program code (Spalter & Dam, 2003), or it can be any digital or non-digital object and used or reused for the technical support of learning (Friesen, 200). Learning objects have some common characteristics (Paulsson & Naeve, 2003): they are described according to learning technology standards of an organization such as Institute of Electrical and Electronics Engineers, Inc. (IEEE)/Learning Technology Standards Committee (LTSC), Instructional Management Systems (IMS), and SCORM; and the content is modularized, which can make each learning object independent and thus increase the opportunities to reuse them. According to Advanced Distributed Learning (200), a learning object should have the following characteristics: Accessibility: the ability to locate and access instructional components from one remote location and deliver them to other locations. Interoperability: the ability to take instructional components developed in one location with one set of tools or platform and to use them in other locations with a diverent set of tools or platform. Durability: the ability to withstand technology changes without redesign, reconwguration or recoding. Reusability: the Xexibility to incorporate instructional components in multiple applications and contexts. If the development of learning objects followed these four characteristics, e-learning would evolve steadily. 2.2. Semantic metadata Metadata take two forms: syntactic and semantic. Syntactic metadata describe non-contextual information about content, and semantic metadata describe domain-speciwc information about the content. The W3C forum proposed Semantic Web, a standard for semantic metadata, which is not only understandable by humans but also analyzable by machines. The Semantic Web is an extension of the current web in which information is given well-dewned meaning, better enabling computers and people to work in cooperation (Berners-Lee, Hendler, & Lassila, 200). Realizing the Semantic Web involves use of ontology, or specifying a conceptualization in organizing the concepts and metadata to annotate and represent content (Sheth et al., 2002). Ontology is a key part in Semantic Web structure, playing the role of diverent knowledge processing, sharing and reusing of diverent applications (Decker, Melnik, Van Harmelen, Fensel, & Klein, 2000). Thus, semantic metadata can improve the quality of searches. Four kinds of methods are used in searching data: syntactic search, metadata search, queryby-example, and navigational search (Chiang, Chua, & Storey, 200). Semantic metadata searching is best able to Wnd the relevant material, and ontology is an important method in capturing semantics. To provide a higher search quality, we describe two ontologies, course ontology and content ontology, for diverent purposes in our proposed system.

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 46 3. The architecture of TMDC In designing TMDC, we adopt the concepts of learning content (Fischer, 200), integration of Agent-based Information Management System (AIMS) authoring architecture (Aroyo et al., 2002), and the process model of course authoring architecture (Klein, Ateyeh, Konig-Ries, & Mulle, 2003) to achieve the goal of reusing teaching template and learning objects. The architecture of TMDC contains four modules (shown in Fig. ): teaching material repository, ontology, course database, and course authoring. Four types of users might interact with TMDC: teachers, e-material providers, domain experts, and ontology designers. Teachers are the end users of this system. e-material providers could be teaching material developers, publishers, or an external teaching material repository. Providers can upload learning objects or teaching templates for teachers by using the teaching material repository module. Ontology designers and domain experts construct and maintain the ontology module according to the domain of teaching material. 3.. Teaching material repository module The teaching material repository module preserves and manages learning objects. Many studies have produced results in this area (Brantner, Enzi, Guth, Neumann, & Simon, 200; Paulsson & Naeve, 2003; Shen, Shi, & Xu, 2002; Yeo & Wong, 2002). The teaching material comes from learning object providers. This module functions like a bookstore to help consumers Wnd the books they want, in this case e-materials. 3.2. Ontology module Ontology can help in designing the structure of teaching material (Fischer, 200). Two ontologies expedite the design process of teaching material: course ontology is used to course database module teaching template teaching material repository module e-material provider teacher course authoring module ontology module domain experts and ontology designer Fig.. Architecture of TMDC.

462 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 describe the term relations among course topics, and content ontology describes the relations among learning objects in a material. Table shows the types of relations among course topics. Fig. 2 is an example of course ontology for the course object-oriented analysis and design. This ontology presents the concept relations for the domain of computer language. Fig. 3 shows the content ontology model described by the notation of entity relation (ER) diagram. The main diverence between course ontology and content ontology is that the latter is used to describe the relations among learning objects in an e-material. The relation types used in course ontology, referred from the relation classiwcations in SCORM, are listed in Table 2. Table Relation types in course ontology Relation name IsPrerequisiteFor IsSuperTopicOf SameAs Explanation Describes whether there is an order between the concepts States super- or subrelations between the concepts Describes concepts with the same meanings Course Topic Information System. System Analysis and Design Ingredient Characteristics System Development Life Program Language.... Cycle Procedure Language Object-Orientation Prototyping Life Cycle Spiral Life Cycle Waterfall Life Cycle Encapsulation Traditional Life Cycle Inheritance Polymorphism SameAs issupertopicof isprerequisitefor Fig. 2. An example of course ontology.

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 463 hastype Relationship Type haslearning Type Learning Object Relationship hasproperty Object Property Learning Object isclassification Of Learning template method hastype Course Ontology Learning template Fig. 3. Content ontology model. Table 2 Relation types in content ontology Relation name Explanation HasProperty Describes that a certain concept is the property of another concept HasType Describes that a certain concept is a type of another concept IsClassiWcationOf Describes that the classiwcation of a certain concept is the type of another concept HasLearningType Describes that a certain concept uses another concept as a kind of teaching method 3.3. Course database module The course database module preserves the teaching material structure that has been developed. It is a knowledge base overing reference for new course development. Reusing or revising the existing course structure requires less time than developing a new course. The course database module includes the teaching objective, structure of course units, teaching template, learning objects used for course content, and course structure. The course structure used in the proposed system is shown in Fig. 4. 3.4. Course authoring module e-material authors need to have relevant background knowledge, create a content outline, and write the content of the article in accordance with our procedure. The following steps are the workxow of the course authoring module: Step : Select course domain. Because ontology is domain dependent, a course author must con- Wrm the discipline the course belongs to for selecting course ontology.

464 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 Course Topic Teaching Method (from Teaching template) Course Content (from Learning object) Course Topic 2 Teaching Method (from Teaching template) Course Content (from Learning object) Course Topic 3 Teaching Method (from Teaching template) Course Content (from Learning object) Course Topic 4 Teaching Method (from Teaching template) Course Content (from Learning object) Fig. 4. Course structure (modiwed from Klein et al., 2003). Step 2: Describe the goal of teaching material. The author must describe what the users can expect to learn. This description is used for searching and for another author to understand the material. Step 3: Refer to similar courses. The purpose of searching existing courses in the course database module is to consult others development achievement and over the chance to reuse the similar course structure. Through the course ontology and the querying rules, the proposed system can help the course author Wnd similar e-material. Step 4: Choose the course table of contents. Developing a new course involves describing the contents of the course. At this stage, the course author should choose the topics to be covered. The course ontology in the ontology module will be used to accelerate this process. Step 5: Design the teaching template. The teaching template structures the learning Xow of a course. For example, while introducing a certain course topic, a teacher prefers Wrst to state why one learns the course topic, then explains dewned terms, and illustrates with examples for understanding the content. The possible design of this teaching template can be seen in Fig. 5. Step 6: ConWrm the course outline. After conwrming the course topics and the teaching templates, the author produces the whole frame of the new course. The course outline is produced by combining the result of Step 4 with that of Step 5. At this stage, the author needs to further conwrm topics or make amendments. Fig. 6 shows an example of a complete course outline. Step 7: Choose the content of teaching material. After conwrming the course outline, the author can select the contents, the necessary learning objects. The proposed system will search and recommend learning objects. An example of this interface can be seen in Fig. 7. Step 8: Revise the content. Because the entire content of the courseware is assembled by learning objects according to the teaching template, the course content is only one piece of semimanufactured product in the proposed system. The author needs to artiwcially inspect, revise, and adjust the course content in this step.

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 465 Motivation Teaching Sequence Definition Example Exercise Solution Fig. 5. An example of the teaching template. Information System. Characteristics.2 Ingredient 2 Object-Orientation 2. Inheritance 2.2 Encapsulation 2.3 Polymorphism 3 System Development Life Cycle 3. Waterfall Life Cycle 3.2 Prototyping Life Cycle Example Definition Exercise Example Defintion Example Definition Example Definition Definition Explaination Definition Explaination Fig. 6. An example of a course outline. 4. Evaluation 4.. Methodology We evaluated TMDC from a functionality and usability perspective. Evaluating the system involves completion of the following tasks: () ontology designers set up the ontology model, by

466 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 Protégé 2000; (2) system designers implement a TMDC prototype system for evaluation; (3) participants use the TMDC to design a course; and (4) analysts assess the experimental results. 4.2. Participants Fig. 7. An example of a list of available course objects. We invited 30 postgraduate students who had been section tutors of the Network Management course. They had participated in designing teaching materials for diverent topics in the network management domain. They used the TDMC system to design teaching material and compared it with the method they had used. 5. Results 5.. Experiment Time to develop a new course by use of TMDC and a traditional method. The evects of the procedure of course authoring is also veriwed. Table 3 shows the time spent to develop a new course with use of the TMDC and a traditional system. Almost half of the participants (43.33%) spent 5 20 min generating their teaching material with use of TMDC. TMDC required a substantially shorter time for course development than the traditional system. Table 4 shows the degree of acceptance of TMDC. Most participants (73.33%) found TMDC to be acceptable. The mean acceptance score was 3.833. Respondents who chose 2 or 3 said that they preferred to check all materials one by one, not ones recommended by a system.

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 467 Table 3 Time to develop a new course Time to develop a new course Number (%) of participants (n D 30) Less than 5 min 2 (6.7%) 5 20 min 3 (43.3%) 20 25 min 5 (6.6%) 25 30 min 6 (20.0%) More than 30 min 4 (3.3%) Mean development time using the TDMC 22.3 min Mean development time using the other methods.58 h Table 4 The degree of acceptance of TMDC Strongly dissatiswed DissatisWed Neutral SatisWed Strongly satiswed Evaluation value 2 3 4 5 Number of participants (%) 0 (0%) (3.3%) 5 (6.6%) 22 (73.3%) 2 (6.66%) Mean evaluation 3.833 5.2. Experiment 2 This evaluation determined whether semantic metadata could give positive results for teaching template design and learning object selection. Fig. 8 shows the recall value from 30 participants who searched existing teaching templates by using the TMDC. The participants used two diverent methods: keyword search and semantic metadata search. The recall values ranged from 0.5 to with semantic metadata search and from 0.2 to 0.85 with keyword search. The second measurement evaluated precision of searches, shown in Fig. 9. Precision with the semantic metadata search was lower than with keyword search. Thus, users who wish fewer but more precise results can choose the keyword search in the TMDC system. Searching existing coursewares.00 recall value 0.80 0.60 0.40 0.20 0.00 3 5 7 9 3 5 7 9 2 23 25 27 29 ID of experiment participants Course Ontology keyword-based Fig. 8. The recall values of searching historical course data.

468 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 Searching existing coursewares precision value 0.8 0.6 0.4 0.2 0 3 5 7 9 3 5 7 9 2 23 25 27 29 ID of experiment participants Course Ontology keyword-based Fig. 9. The precision values of searching existing courseware. Previous evaluation showed that even if participants found related courseware on a topic, the learning objects might not have been reusable. Therefore, we evaluated the reusability of learning objects suggested by the proposed system. All participants responded that many existing learning objects recommended by the proposed system had been useful. The recall and precision values of searching learning objects are shown in Figs. 0 and, respectively. Compared with the traditional way to search learning objects with use of keywords, the ontology-based search can give higher recall and precision values. From this experiment, we have not only shown that in the past, learning objects were repeatedly designed, but also indirectly veriwed that the proposed search method can Wnd reusable, existing teaching materials. Table 5 presents the results of the eyciency of reusing teaching materials. Most participants (93.33%) were satiswed with the evectiveness of reusing the teaching materials found with TMDC. The mean satisfaction was 4.33. Thus, our system provides ease in developing material for courses. Searching Learning Object recall value 0.8 0.6 0.4 0.2 0 3 5 7 9 3 5 7 9 2 23 25 27 29 ID of experiment participants Ontology-based keyword-based Fig. 0. The recall value distribution of searching for learning objects.

H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 469 Searching Learning Object 0.8 0.6 0.4 0.2 0 3 5 precision value 7 9 3 5 7 9 2 23 25 27 29 ID of experiment participants Ontology-based keyword-based Fig.. The precision value distribution of searching for learning objects. Table 5 EYciency of reusing teaching materials by TMDC Strongly dissatiswed DissatisWed Neutral SatisWed Strongly satiswed Evaluation value 2 3 4 5 Number of participants (%) (n D 30) 0 (0%) 0 (0%) 2 (6.7%) 22 (73.3%) 6 (20%) Mean evaluation 4.33 6. Conclusion e-learning is a trend in education. Learners can benewt because the technology has no restriction in time and distance. However, building an e-learning environment means not only building a system but also generating high-quality teaching materials. DiVerent teachers may not need the content of a whole package of e-material from a single author. Many prefer to design teaching material by combining several learning objects. Authors spend a lot of time designing similar teaching materials repeatedly. How to give the provider and consumer a convenient environment for producing and selecting teaching material is important. In this paper, we describe our TMDC system, which allows for such an environment. Moreover, we recommended a course authoring procedure to generate teaching material. The evaluation results show that the authoring procedure is acceptable and courseware authors can Wnd existing materials to facilitate their authoring processes. References Advanced Distributed Learning (200). SCORM version.2 the SCORM content aggregation model.<http:// www.adlnet.org/adldocs/documents/scorm_.2_cam.pdf>200 Retrieved October, 2003. Aroyo, L., Dicheva, D., & Cristea, A. (2002). Ontological support for web courseware authoring. Intelligent Tutoring Systems, Lecture Notes in Computer Science, 2363, 270 280.

470 H.-C. Wang, C.-W. Hsu / Computers & Education 46 (2006) 458 470 Berners-Lee, T., Hendler, J., & Lassila, O. (200). The Semantic Web. ScientiWc American, 7(May), 34 43. Bohl, O., Schellhase, J., Sengler, R., & Winand, U. (2002). The sharable content object reference model (SCORM) a critical review. In Proceedings of the international conference on computers in education (ICCE 02), Auckland, New Zealand. Brantner, S., Enzi, T., Guth, S., Neumann, G., & Simon, B. (200). Universal-design and implementation of a highly Xexible E-Market-Place for learning resources. In Proceedings of the IEEE international conference on advanced learning technologies, Madison (WI), USA. Chiang, R. H. L., Chua, C. E. H., & Storey, V. C. (200). A smart web query method for semantic retrieval of web data. Data & Knowledge Engineering, 38, 63 84. Decker, S., Melnik, S., Van Harmelen, F., Fensel, D., & Klein, M. (2000). The Semantic Web: the role of XML and RDF. IEEE Internet Computing, 4(5), 63 74. Fischer, S. (200). Course and exercise sequencing using metadata in adaptive hypermedia learning systems. Journal on Educational Resources in Computing, (). Friesen, N. (200). What are educational objects? Interactive Learning Environments, 9(3), 29 230. Klein, M., Ateyeh, K., Konig-Ries, B., & Mulle, J. (2003). Creating, Wlling, and using a repository of reusable learning objects for database courses. In Proceedings of BTW-workshop in E-Learning (BTW-EL03). Germany: University Leipzig. Paulsson, F., & Naeve, A. (2003). Standardized content archive management SCAM storing and distributing learning resources. IEEE Learning Technology Newsletter, 5(), 40 42. Shen, Z., Shi, Y., & Xu, G. (2002). A learning resource metadata management system based on LOM speciwcation. In Proceedings of the 7th international conference on computer supported cooperative work in design (CSCWD 2002), Rio de Janeiro, Brazil. Sheth, A., Bertram, C., Avant, D., Hammond, B., Kochut, K., & Warke, Y. (2002). Managing semantic content for the web. IEEE Internet Computing, 6(4), 80 87. Silva, V. T., Lucena, C. J. P., & Fuks, H. (200). ContentNet: a framework for the interoperability of educational content using standard IMS. Computers & Education, 37(3 4), 273 295. Spalter, A. M., & Dam, A. (2003). Problems with using components in educational software. Computers & Graphics, 27, 329 337. Yeo, G. K., & Wong, P. Y. (2002). Organized exchange repositories for learning resources. In Proceedings of the international conference on computers in education (ICCE 02), Auckland, New Zealand.