Paper Symposium Integrative Statement (word count = 247) Using Cognitive Science to Inform Mathematics Instruction The call for improvement in our country's mathematics education is strong. Increasingly, education policy makers are looking towards evidence-based sources for guidance as to what works to improve children's understanding of mathematics. The cognitive sciences have significant contributions to make in the field of educational research, not only in theories of learning and memory, but also in experimental methodologies. Different mathematics instructional methods influenced by cognitive science principles are evaluated experimentally in the four papers presented in this symposium. The first paper manipulates instruction in a 1-to-1 number board game with kindergarteners. The experimental condition made larger gains as was predicted by a theoretical analysis of the instruction. The second paper explores the role of perceptual learning in extracting the structure of word problems for elementary students in both home-schooled and classroom environments. Training that combined solving with perceptual learning improved students ability to extract relevant features of word problems and contributed to improved performance when solving novel word problems compared to solving alone. Paper three presents two classroom level experiments that integrate correct and incorrect worked examples with self-explanation into Algebra 1 classroom lessons. These experiments indicate that simply providing example-based assignments to Algebra students does not always lead to better performance in classroom settings. Paper four investigates the role of self-explanation and amount of practice and time on the learning of mathematical equivalence in elementary school students. These four papers investigate the important topic of mathematics instruction using experimental methodologies guided by cognitive science principles.
Paper Symposium Abstract (word count = 487) Paper 1: Making Numerical Board Games Even Better Through Cognitive Analysis Understanding numerical magnitudes is foundational for effective mathematics learning. Performance on two common measures of numerical magnitude knowledge numerical magnitude comparison and number line estimation correlates strongly with mathematics achievement test scores at all grade levels from kindergarten through eighth grade (Booth & Siegler, 2006; Geary, et. al, 2008; Holloway & Ansari, 2009; Schneider, Grabner, & Paetsch, 2009). One contributor to knowledge of numerical magnitudes is playing linear number board games such as Chutes and Ladders (Ramani & Siegler, 2008; Siegler & Ramani, 2008; 2009; Whyte & Bull, 2008). How the physical and instructional and features of board games combine to produce learning of numerical information is poorly understood, however. In the present study, we tested a general theoretical perspective on instructional design: the cognitive alignment hypothesis. We hypothesized that aligning children s learning activities with a desired mental representation of numerical magnitudes would increase the children s numerical skills and knowledge. More specifically, we examined whether a feature of instruction that was hypothesized to influence encoding and learning of numerical information type of counting during game playing does in fact influence encoding and learning. We randomly assigned kindergartners (N = 42) to play the same 0-100 board game in one of two instructional conditions that differed in what children said as they moved their token. In the count-from-one condition, children counted aloud from 1 as they moved their token (e.g., children on 17 who spun a 2 said 1 as they put their token on the square labeled 18 and 2 as they put their token on the square labeled 19). In the count-on condition, children counted-on from the number on the square where they began the turn (e.g., children on 17 who spun a 2 said 18, 19 ). Children in each condition played the game 8 times over two weeks. Pretests and posttests included three tasks intended to measure numerical knowledge (number line estimation, numeral identification, and counting-on) and two tasks designed to measure encoding of the game board (game board reproduction and numeral position encoding). As predicted, having kindergartners play a 0-100 number board game in a manner in which their verbal, physical, and mental activities were aligned to the linear numerical features of the board game was crucial to learning. Counting-on led to improvements in number line estimation, numeral identification, counting from numbers other than one, and encoding of the structure of the game board that were roughly twice as great as for children who counted from one while playing the game (Figure 1). Moreover, the count-on procedure produced a catching-up effect, in which children with below average initial numerical knowledge learned more from playing the board game than peers who started with greater knowledge (Figure 2). The idea that the type of counting would have such a large and pervasive effect on learning would have been unlikely without the cognitive alignment hypothesis and the analysis of how learning occurred within the board game context.
Count-from-1 Count-on (a) Number Line Linearity (b Number Line Slope (c) Numeral Identification (d Counting-On (e) Numeral Position Encoding (f) Game Board Figure 1. Pretest and posttest performance of children in each condition for each outcome measure
Below Median on Pretest Above Median on Pretest (a) Number Line Linearity (b) Number Line Slope (c) Numeral Identification (d) Counting-On (e) Numeral Position Encoding (f) Game Board Reproduction Figure 2. Pretest and posttest performance of children in the count-on condition who scored above or below the median on pretests
Paper Symposium Abstract (word count = 500) Paper 2: Perceiving Structure in Word Problems: Applying Perceptual Learning to Elementary Math Pedagogy Mathematics education should be the study of structure, but U.S. math education overemphasizes the limited notion of solving problems based on rules (i.e., Stigler & Hiebert, 2009). The assumption inherent in this pedagogy is this: by learning to solve, students will (at least implicitly) understand structure. It is unclear whether an emphasis on solving leads to an appreciation of structure. Also, numerous studies in nonmathematical domains have shown that learners are able to extract structural information if given the opportunity to categorize multiple instances that vary in superficial ways but are constant in their underlying structure (e.g., Gibson, 1969). In two studies, we examine the following questions: (1) If given appropriate opportunities, can students employ this general mechanism of extraction, called perceptual learning, to learn about mathematical structure underlying word problems? (2) Does better solving mean better structure identification? In Study 1, 31 third-grade homeschooled students from various states participated in our program (called Math Problem Insight, MPI) through an online learning provider. Students were shown a variety of single-step word problems only containing integers that can be solved with addition, subtraction, multiplication, or division. These problems were presented in two types of short interactive trials: solving and structure-mapping. In solving trials, students were shown word problem and asked to enter a numerical answer. In structure-mapping trials, students were shown similar word problems, but were asked to choose the symbolic expression or short sentence that corresponds to the structure of the situation from four presented options (Figure 1a). Solving and mapping trials were evenly mixed, and structural feedback was shown whenever an incorrect answer was entered/chosen (Figure 1b). Pre/posttests containing novel word problems were administered before/after MPI. Training that combined solving with perceptual learning through mapping across multiple isomorphic representations improved students' pick up of abstract structure. Both mapping and solving improved significantly after training (Table 1) shown by accuracy increases and RT decreases. Throughout MPI, students were significantly better at solving problems than representing structure even when they simply have to identify the operation used to solve the problem (see Figure 1a). Previous math learning, that emphasized solving, did not produce equivalent levels of solving and structure appreciation. In Study 2, we tested whether extracting structures of integer word problems would transfer to structurally equivalent word problems containing large integers, fractions, and decimals. A class of fifth-grade students took a paper-and-pencil preand posttest where they attempted to solve word problems that involved large integers (>1000), fractions, and decimals. They also completed three days of MPI in the computer lab. Students showed significant improvement though the test problems substantially differed from the online intervention. Improvements were especially pronounced for large integer and decimal problems, however fraction problems showed no improvement. This set of studies suggests that students are able to extract relevant features of word problems through perceptual learning, presumably contributing to their improved performance on novel word problems. Also, a focus on solving has not produced deep structural understanding, a skill necessary for a flexible understanding of mathematics.
Figure 1: (A) Screenshot of a Mapping trial showing four possible options. (B) The animated feedback shown if the chosen option is incorrect.
Solving Mapping Accuracy RT Accuracy RT N Study 1 Pretest.81 (.04) 32 (3).63 (.04) 33 (3) 31 Posttest.89 (.02) 21 (2).75 (.03) 19 (2) 31 Study 2 Pretest.45 (.08) 16 Posttest.54 (.09) 16 Table 1: Proportion correct and RTs on pre- and posttest measures from Study 1. Proportion correct from Study 2 (no RTs available because these were paper-and-pencil tests). Standard errors are shown in parentheses.
Paper Symposium Abstract (word count = 473) Paper 3: Testing the Worked Example Principle in Real-World Classrooms Numerous studies have shown that students learn better with a combination of examples and problems during practice than practice problems alone. Thus, this technique comes highly recommended for classroom use by the US Department of Education. However, surprisingly few studies have tested the worked example effect in traditional, real-world classroom contexts. This study evaluates worked examples with prompts for self-explanation in Algebra 1 assignments over the course of one unit (Experiment 1) and the entire course (Experiment 2). We utilized a combination of correct and incorrect examples in this study, as this has been shown to be more effective than correct examples alone. In both experiments, example-based assignments were expected to yield better posttest scores than traditional, problem-based assignments. Individual differences in benefit from example-based assignments were also examined. Both example-based and control (problem-based) assignments (Figure 1), as well as all assessments were developed in collaboration with the Strategic Education Research Partnership Institute (SERP) and teachers from Minority Student Achievement Network (MSAN) school districts. In Experiment 1, we randomly assigned individual students (N=51) in three classrooms to receive 4 example-based (N=26) or 4 control assignments (N=25) during their one-month unit on solving linear equations. Conceptual and procedural knowledge were measured before and after the unit. Example-based assignments were more effective for both types of knowledge; example-based assignments were especially beneficial for African-American and Hispanic students than Caucasian and Asian students. These assignments also closed the achievement gap (Figure 2). In Experiment 2, 73 classrooms in 5 MSAN districts were randomly assigned to the examplebased (N=37) or control group (N=36). The sample included 1014 students (492 examplebased, 522 control). At the beginning of the school year, students were tested on prerequisite knowledge for Algebra. During the school year, teachers used their designated type of study assignments in place of assignments they would typically use. Teachers were encouraged to use all 24 assignments of their designated type, with approximately one study assignment used each week. At year-end, students were tested on 10 Algebra-related released items from standardized tests used by the districts. Non-minority students benefitted more from examplebased assignments than did minority students, which was opposite of Experiment 1. However, students attempted fewer examples than problems, and attempting more examples was associated with higher test scores, even after accounting for prior knowledge. Further, minority students were also less likely to try to solve practice problems on example-based assignments; greater attempts of those problems were also related to higher scores. Simply providing example-based assignments to Algebra students does not necessarily lead to better performance; more ecologically valid classroom studies are necessary to determine when the technique is useful. Perhaps the key to successful example-based practice in the classroom is to present it in a way that encourages persistence through the exercises and drawing connections between the examples and problems to make student problem-solving a deeper experience.
Figure 1: Excerpts from the example-based (a) and control (b) versions of the assignment on Writing Expressions and Equations from Words a. Example-based b. Control
Figure 2: Experiment 1 Achievement gap in Control and Example-Based groups for composite scores.
Paper Symposium Abstract (word count = 499) Paper 4: Is the Benefit of Self-Explanation Simply Added Time on Task? Improving student learning is a goal that many people hold dear. One method for achieving this goal that has empirical backing is prompting students to generate explanations to themselves in an attempt to make sense of new information, also known as the self-explanation effect (Chi, 2000). The effect of this intervention has been proven in several domains, including mathematics (e.g., Wong, Lawson & Keeves, 2002). The general paradigm used to test the self-explanation effect is to have two groups solve the same problem set, but one group self-explain as they do so. However, an artifact in such studies is that the self-explain group spends more time overall on the problem set. A handful of studies have addressed this issue by controlling for the amount of time spent on the learning task between groups. Some studies have found self-explaining to be beneficial (e.g., Aleven & Koedinger, 2002), while others have not (e.g., GroBe & Renkle, 2003). However, these mixed results may be partially due to past studies failing to incorporate control groups that take amount of time and amount of practice into account. The current study investigated the effects of self-explanation when learning mathematical equivalence with control groups matched for the amount of time on task and amount of practice. Understanding mathematical equivalence entails understanding that the equal sign means that two sides of an equation are the same, and is embodied in an ability to solve equations such as 4 + 2 +3 = + 6 (McNeil, 2008). One hundred ten students in grades 2-4 worked through a tutoring session in one of three conditions. Students were administered a pretest, posttest, and two week retention test that assessed procedural and conceptual knowledge (Table 1). All students first received a brief lesson on solving mathematical equivalence problems. In the Practice 1 (Pr1) condition, students (N = 31) then solved six problems. In the Self-Explain condition (SE), students (N = 31) solved the same six problems, but were prompted to selfexplain the correct and incorrect solutions to each problem. The Practice 2 (Pr2) condition had students (N = 36) solve twelve problems so that they spent approximately the same amount of time on the intervention as the SE condition. Benefits of self-explanation were determined by comparing the SE group against the amount of practice control condition, and the time on task control condition. Differences between conditions were evaluated with repeated measures ANCOVA models, with post and retention test scores as dependent measures and pretest scores, backward digit span score, and grade as covariates. Preliminary analyses indicated a main effect for condition on procedural knowledge. See Table 2 for mean scores. In particular, students in the SE condition had higher procedural knowledge scores than students in the Pr1 condition. Procedural knowledge for students in the Pr2 condition was in between these two conditions and did not differ significantly from either one. These results suggest that self-explanation was beneficial, but this may be partially due to the amount of time on task.
Knowledge Type Example Item Conceptual Procedural What does the equal sign mean? Find the number that goes in the blank: 7 + 6 + 4 = 7 + Table 1: Example assessment items that tap conceptual and procedural knowledge of mathematical equivalence.
Conceptual Knowledge Procedural Knowledge Condition Mean SD Mean SD Pr1 45.9 27.1 58.8 32.2 Pr2 45.0 25.2 59.9 28.3 SE 45.7 23.5 67.1 30.5 Table 2: Average post and retention test scores of conceptual and procedural knowledge of mathematical equivalence by condition.