Knowledge Spaces
Jean-Claude Falmagne Dietrich Albert Christopher Doble David Eppstein Xiangen Hu Editors Knowledge Spaces Applications in Education
Editors Jean-Claude Falmagne School of Social Sciences, Dept. Cognitive Sciences University of California, Irvine Irvine, CA, USA Christopher Doble ALEKS Corporation Irvine, CA, USA Dietrich Albert Department of Psychology University of Graz Graz, Austria David Eppstein Donald Bren School of Information & Computer Sciences University of California, Irvine Irvine, CA, USA Xiangen Hu Department of Psychology University of Memphis Memphis, TN, USA ISBN 978-3-642-35328-4 ISBN 978-3-642-35329-1 (ebook) DOI 10.1007/978-3-642-35329-1 Springer Heidelberg New York Dordrecht London Library of Congress Control Number: 2013942001 Springer-Verlag Berlin Heidelberg 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Preface As can be gathered from the database maintained by Cord Hockemeyer at the University of Graz 1, the literature on Knowledge Spaces is expanding rapidly. The recent book by Falmagne and Doignon (2011) gives a comprehensive account of the theory up to its date of publication. However there have been important developments after that. We thought that a volume gathering some of these developments would be timely. This volume has two parts. Part I describes a number of chosen empirical works. Part II deals with recent theoretical results. Two chapters play a special role. The first chapter in Part I, called Overview, gives an informal, intuitive presentation of all the important concepts of learning space theory, which is an important special case of knowledge space theory. This chapter is intended for readers primarily interested in the applications of learning space theory. Chapter 8, the first chapter in Part II, gives a technical presentation of the theory, including all the main theorems (without the proofs) and a description of some important algorithms. This chapter may serve as the basic reference for the works described in Part II. In Part I, Chapters 2 to 6 describe some applications of the ALEKS system in education. The ALEKS system is the most elaborate application of learning space theory to date. It is equipped with an assessment module and a learning module. This system is bilingual (English and Spanish) and currently covers all of K 12 mathematics 2 (excluding calculus), and beginning chemistry. Using extensive assessment and learning data, Chapter 2 investigates various statistical measures of the validity of the assessment and learning modules of the ALEKS system. Chapter 3 reports an application of the ALEKS system at the University of Illinois. The chapter demonstrates that the use of ALEKS as a placement test replacing the SAT and the ACT resulted in a substantial decrease of F grades and withdrawals. Chapter 4 describes two studies evaluating whether online individualized instruction by the ALEKS system will increase student scores on a standardized high-stakes test. (The answer is Yes. ) The students were elementary and middle school students of the Memphis area schools. The ALEKS system was used at the University of Memphis to teach a statistics course to students majoring in psychology or social sciences. Chapter 5 compares the results for black and white students in this course with those obtained in a traditional lecture type course. While the white students, on the 1 See: http://liinwww.ira.uka.de/bibliography/ai/knowledge.spaces.html. 2 Some of these subjects are also used in colleges and universities. V
VI Preface average, do much better than the black students in the lecture type course which is consistent with traditional results the discrepancy disappears in the online course. Chapter 6 describes an application of the ALEKS system to the teaching of General Chemistry to college students. The chapter presents a selection of learning and assessment data for the course, and gives interpretations of those data in the framework of knowledge space theory. Chapter 7 analyzes the ability of students to make logical connections between fundamental chemical principles and the various representations of chemical phenomena. The ALEKS system is not used in this study. The authors build their own knowledge structures. As mentioned earlier, the introductory chapter of Part II, Chapter 8, gives a condensed description of the most important mathematical results. Distinguishing between behavioral performance and its underlying skills and competencies, Chapters 9 and 10 deal with performance while Chapters 11 and 12 focus on competencies; all of them go well beyond Albert and Lukas (1999). Chapter 9 describes recent extensions in knowledge space theory (multiple answer alternatives, relations between sets/tests), relationships between knowledge space theory and other theoretical approaches (formal concept analysis, FCA; latent class analysis, LCA; item response theory, IRT) as well as methods for data driven generation of knowledge structures, their empirical validation (item tree analysis, ITA; inductive item tree analysis, IITA; measures and indices of t) and respective software resources. Methodological considerations and applications in Chapter 10 exemplify empirical research dealing with generating and validating knowledge structures for sets of items or tests. The different skill- and competence-oriented approaches have been developed independently. Thus, Chapter 11 for the first time relates systematically these approaches to each other by presenting a united framework which allows for identifying their commonalities and differences. These approaches are further developed in Chapter 12, which asks how to deal with distributed information, how to formulate a probabilistic approach, how to link observable navigation and problem solving behavior to cognitive and competence states, how to support self-regulated learning behavior, and how to assess competencies in educational games noninvasively. Furthermore, respective applications in technology enhanced learning and competence management are described. Chapters 13 and 14 describe a data structure, the learning sequence, that can be used to efficiently implement learning-space based systems. In Chapter 13, learning sequences are applied to the tasks of generating the states of a learning space and using learning spaces to assess the knowledge of a learner. Chapter 14 discusses the use of learning sequences to project learning spaces onto smaller sets of concepts (important for the efficiency of assessment in large learning spaces) and to modify learning spaces by adding or removing states.
Preface VII We are most grateful to all the referees whose reports led to improvements of the presentation of the works described in this volume. We thank in particular Eric Cosyn, Cornelia Dowling, Yung-Fong Hsu, Mathieu Koppen, Jeff Matayoshi, Alexander Nussbaumer, Martin Schrepp, Luca Stefanutti, Reinhard Suck, and Nicolas Thiéry. We also thank Brian Junker and Don Laming for their useful reactions to a presentation of the material in Chapter 2. The Editors March 11, 2013.
Contents Part I LEARNING IN A KNOWLEDGE SPACE 1 Overview... 3 Jean-Claude Falmagne and Christopher Doble 1.1 Introduction... 3 1.2 BasicConcepts:KnowledgeStatesandSpaces... 5 1.3 UncoveringaKnowledgeState... 13 1.4 TheCaseofVeryLargeKnowledgeSpaces... 18 1.5 HistoricalNote... 21 1.6 AgraphofaLargerStructure... 22 1.7 NontechnicalGlossary... 23 2 Assessing Mathematical Knowledge in a Learning Space... 27 Eric Cosyn, Christopher Doble, Jean-Claude Falmagne, Arnaud Lenoble, Nicolas Thiéry and Hasan Uzun 2.1 Introduction... 27 2.2 Measuring the Validity/Reliability of an Assessment..... 28 2.3 StatisticalMethods... 29 2.4 DataAnalysis... 36 2.5 SummaryandDiscussion... 45 2.6 Appendix... 49 3 ALEKS-based Placement at the University of Illinois... 51 Alison Ahlgren Reddy and Marc Harper 3.1 Introduction... 51 3.2 PlacementattheUniversityofIllinois... 52 3.3 The University of Illinois Math Placement Program...... 52 3.4 Description of the Data.... 53 IX
X Contents 3.5 AnalysisofData... 56 3.6 Discussion... 64 4 The Impact of a Mathematical Intelligent Tutoring System on Students Performance on Standardized High-Stake Tests... 69 Jeremiah Sullins, Rob Meister, Scotty D. Craig, William M. Wilson, Anna Bargagliotti and Xiangen Hu 4.1 Introduction... 69 4.2 ALEKSinMemphisAreaSchools... 72 4.3 Study1... 72 4.4 Study2... 75 4.5 SummaryandDiscussion... 78 5 A Potential Technological Solution for Reducing the Achievement Gap Between White And Black Students... 79 Xiangen Hu, Yonghong Jade Xu, Charles Hall, Kristin Walker and Theresa Okwumabua 5.1 Introduction... 79 5.2 Narrowing the Achievement Gap through Technology in the Classroom... 81 5.3 Objective... 82 5.4 Methods... 83 5.5 Results... 85 5.6 Summary... 90 6 A Commercial Implementation of Knowledge Space Theory In College General Chemistry... 93 Christopher J. Grayce 6.1 Introduction... 93 6.2 ALEKS GenChem... 95 6.3 Results...103 7 Using Knowledge Space Theory to Assess Student Understanding of Chemistry...115 Mare Taagepera and Ramesh D. Arasasingham 7.1 Introduction...115 7.2 ExperimentalDesign...117 7.3 DeterminationofKnowledgeStructure...120 7.4 AnalysisandDiscussion...121 7.5 Conclusion...127
Contents XI Part II RECENT THEORETICAL PROGRESS 8 Learning Spaces: A Mathematical Compendium...131 Jean-Paul Doignon, Jean-Claude Falmagne and Eric Cosyn 8.1 AxiomsforLearningSpaces...132 8.2 TheBaseandtheAtoms...135 8.3 TheFringeTheorem...136 8.4 ProjectionsofaKnowledgeStructure...137 8.5 BuildingaLearningSpace...138 8.6 Probabilistic Extension...140 8.7 The Assessment Algorithm...140 8.8 AboutPracticalImplementations...143 9 Recent Developments in Performance-based Knowledge Space Theory...147 Ali Ünlü, Martin Schrepp, Jürgen Heller, Cord Hockemeyer, Gudrun Wesiak and Dietrich Albert 9.1 ExtensionsofKnowledgeSpaceTheory(KST)...147 9.2 Relationships between KST and other Theoretical Approaches 157 9.3 Methods for Construction and Validation of Knowledge Structures...171 9.4 Applications...185 9.5 Discussion...190 10 Generating and Validating Relations between Sets/Tests..193 Dietrich Albert, Gudrun Wesiak and Ali Ünlü 10.1 Introduction...193 10.2 Basic Concepts of SRbT and Corresponding Test Knowledge Spaces...194 10.3 Generation of Hypotheses......197 10.4 Validation of Hypotheses...211 10.5 Discussion...224 10.6 ListofAbbreviations...227 11 Skills, Competencies and Knowledge Structures...229 Jürgen Heller, Ali Unlü and Dietrich Albert 11.1 Introduction...229 11.2 Relating Skills to Knowledge Structures....230 11.3 Relating Knowledge Structures to Skills....235 11.4 Korossy scompetence-performanceapproach...236 11.5 ExtendingtheCompetence-PerformanceApproach...240 11.6 Conclusions...241
XII Contents 12 Recent Developments in Competence-based Knowledge Space Theory...243 Jürgen Heller, Thomas Augustin, Cord Hockemeyer, Luca Stefanutti and Dietrich Albert 12.1 Introduction...243 12.2 Distributed Skill Functions and the Meshing of Knowledge Structures...245 12.3 Probabilistic Competence Approach...253 12.4 Linking Navigation Behavior to the Cognitive State in elearning...260 12.5 Micro-Adaptivity: Non-invasive Skill Assessment in Educational Games......270 12.6 ApplicationsinLearningandCompetenceManagement...278 12.7 GeneralConclusionsandDiscussion...285 13 Learning Sequences: An Efficient Data Structure for Learning Spaces...287 David Eppstein 13.1 Overview...287 13.2 LearningSpacesandQuasi-OrdinalSpaces...289 13.3 From State Generation to Assessment....290 13.4 LearningSpacesfromPartialOrders...292 13.5 LearningSpacesfromLearningSequences...296 13.6 Representing States by Numerical Vectors...299 13.7 GeneratingtheStatesofaLearningSpace...301 13.8 FutureWork...303 13.9 Conclusions...304 14 Projection, Decomposition, and Adaption of Learning Spaces...305 David Eppstein 14.1 Projection...305 14.2 FibersofProjections...310 14.3 FindingConciseSetsofLearningSequences...314 14.4 DecompositionofLearningSpaces...317 14.5 AdaptingaLearningSpace...318 14.6 FutureWork...321 14.7 Conclusions...322 Bibliography...323 Index...345