Scaffolding Students' Reflection for Science Learning. Elizabeth Anna Davis

Transcription

1 Scaffolding Students' Reflection for Science Learning by Elizabeth Anna Davis B.S.E. (Princeton University) 1989 M.A. (University of California, Berkeley) 1994 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Education in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Marcia C. Linn, Chair Barbara Y. White Michael J. Clancy Spring 1998

2 The dissertation of Elizabeth Anna Davis is approved: Chair Date Date Date University of California, Berkeley Spring 1998

3 Table of Contents List of Figures... List of Tables... Acknowledgments... PART ONE Chapter 1: Introduction Introduction and Design... Knowledge Integration... Knowledge Integration and Conceptual Change... Rationale for Reflection Prompts... The Scaffolded Knowledge Integration Framework and Prompting... Hypotheses and Results... Investigating Prompts for Reflection... Investigating Focus of Reflection... Investigating Students' Beliefs... Structure of the Dissertation... Chapter 2: Research Framework Introduction and Rationale... Research on Knowledge Integration... Research in the CLP/KIE Classroom... Research in Programming Classes... The Dissertation Research... Background... Reflection and Reflection Prompts... Students' Beliefs about Science and Learning... Design of Guidance in Technology-Based Learning Environments... Chapter 3: Preliminary Studies Introduction and Rationale... Study 1... Methods... Results: Justification and Synergy... Study 2... Methods... Results: Project Completion and Principled Knowledge Integration... Study 3... Methods... Results: Prompt Response Characterization... Discussion...

4 Chapter 4: Methods Introduction... Research Context... The Research Paradigm... The Students... The Curriculum... The Software... The Assessments... Study Design... Prompting Conditions... Pairing Students... Data Sources and Outcome Measures... Investigations and Analyses... Effects of Reflection Prompts... Focus of Reflection... Beliefs about Science and Learning Science... Synthesis... PART TWO Chapter 5: Effects of Reflection Prompts Introduction and Rationale... Methods... Data Sources... Outcome Measures... Analyses... Project Quality Results... General Project Quality Measures... Critique Measures... Knowledge Integration Measures... Quiz Results... Relationships among Measures and Productivity of Experiences... Summary and Implications... Chapter 6: The Focus of Reflection Introduction and Rationale... Methods... Data Sources and Outcome Measures... Analyses... Reflection Prompts and Project Quality Results... Reflection in Response to Reflection Prompts... Prompt Responses and Project Characteristics... Summary and Implications...

5 Chapter 7: The Role of Students' Beliefs about Science and Learning Introduction and Rationale... Methods... Data Sources... Outcome Measures... Analyses... Results... Characterizing "Productive" and "Less Productive" Beliefs... Comparing Pre- and Post-test Results... Relating Epistemological Dimensions... Relating Beliefs to Performance... Relating Beliefs and Reflection... Comparing Males' and Females' Beliefs... Discussion... Summary and Implications... Chapter 8: Synthesizing the Roles of Prompts, Reflection, Cites, and Beliefs Introduction and Rationale... Methods... Results... Predicting the Quality Measures... Identifying Groups of Students... Investigating Groups of Students... Discussion and Implications... Overview of Reflection Prompts' Effects... Relating Autonomy, Principle-citing, and Elaboration... Considering Prompt Condition and Lack of Reflection... Investigating the Complex Role of Reflecting on Content... PART THREE Chapter 9: Discussion and Conclusions Introduction and Rationale... Discussion... Expanding the Repertoire of Ideas... Identifying Weaknesses in Knowledge... Autonomous Reflection... Synthesis... General Findings... Implications... Future Directions... Prompting, Reflection, and Knowledge Integration... Beliefs... Technology... References...

6 APPENDIXES Appendix A: The "All The News" Article (Study 2)... Appendix B: Activity Prompts "Letter" Document (Study 2)... Appendix C: Self-Monitoring Prompts "Letter" Document (Study 2)... Appendix D: Prompts Used in "All The News" (Study 3)... Appendix E: All The News Article (Dissertation Study)... Appendix F: Evidence (Dissertation Study)... Appendix G: Student Work... Appendix H: Relevant Questions on Beliefs Test... Appendix I: Prompts in All The News (Dissertation Study)... Appendix J: Interview 1... Appendix K: Interview 2... Appendix L: Interview 3... Appendix M: Questions from Pre-, Post-Project Quiz (Dissertation Study)... Appendix N: Examples of Levels of Coherence... Appendix O: Trends in the Data...

7 List of Figures Figure 2Ð1: Guidance in the Knowledge Integration Environment... Figure 3Ð1: Instructions for Clothing Design for Two Conditions in Study 1... Figure 3Ð2: Examples of Prompts... Figure 3Ð3: Instructions for Letter for the Two Conditions in Study 2... Figure 3Ð4: Principled Knowledge Integration... Figure 6Ð1: Reflection Focus for Directed and Generic Prompts... Figure 6Ð2: Reflection Focus for Planning, Monitoring, Generic Prompts... Figure 7Ð1: Mean Dimensions Scores on Pre- and Post-Tests... Figure 7Ð2: Mean Change in Autonomy Scores for each Autonomy Group... Figure 7Ð3: Mean Change in Strategy Scores for each Strategy Group... Figure 7Ð4: Mean Change in Process Scores for each Process Group... Figure 8Ð1: Mean Values of Coherence for Autonomy Groups... Figure 8Ð2: Mean Values of Coherence for Autonomy Groups by Condition... Figure 8Ð3: Mean Values of Coherence for Principle-Citing Groups... Figure 8Ð4: Mean Values of Overall Critique Quality for Elaboration Groups by Condition... Figure 8Ð5: Mean Values of Guidelines Quality for Uncodable Groups by Condition... Figure 8Ð6: Mean Values of Overall Critique Quality for Non-Reflective Groups by Condition... Figure OÐ1: Mean Values of Coherence for Principle-Citing Groups by Condition... Figure OÐ2: Mean Values of Guidelines Quality for Principle-Citing Groups by Condition... Figure OÐ3: Mean Values of Critique Quality for Uncodable Groups by Condition...

8 List of Tables Table 2Ð1: Dimensions of Student Beliefs... Table 3Ð1: Coding Prompt Responses... Table 3Ð2: Characteristics of Comments in Prompt Responses... Table 4Ð1: The Directed and Generic Prompt Conditions... Table 4Ð2: Pairing Students based on Beliefs... Table 5Ð1: Outcome Measures for Student Projects... Table 5Ð2: Class Periods' Project Scores... Table 5Ð3: Significant Differences in Class Periods' Project Scores... Table 5Ð4: Quality of Guidelines... Table 5Ð5: Mean Number of Cites in Claim Notes (Claims 1 and 3)... Table 5Ð6: Mean Number of Cites in Letters... Table 5Ð7: Coherence of Students' Ideas... Table 5Ð8: Correlations among Major Project Quality Measures... Table 5Ð9: Correlations between Principle Cites, Project Quality Measures... Table 6Ð1: Coding Reflection in Prompt Responses... Table 6Ð2: Exemplars of Reflection Types... Table 6Ð3: Degree of Elaboration of Reflection Prompt Responses... Table 6Ð4: Mean Proportion of each Focus of Reflection... Table 7Ð1: Dimensions of Beliefs Investigated... Table 7Ð2: Descriptive Statistics for the Dimensions... Table 7Ð3: Correlations for Pre- and Post-Test Scores... Table 7Ð4: Correlations between Dimensions... Table 7Ð5: Correlations for Dimensions and Performance... Table 7Ð6: Correlations for Pairs' Beliefs and Project Quality Measures... Table 8Ð1: Regression Coefficients for Predicting Overall Critique Quality for Directed Prompt Condition... Table 8Ð2: Regression Coefficients for Predicting Overall Critique Quality for Generic Prompt Condition... Table 8Ð3: Predicting Overall Critique Quality for Directed and Generic Prompts... Table 8Ð4: Regression Coefficients for Predicting Coherence for Directed Prompt Condition... Table 8Ð5: Regression Coefficients for Predicting Coherence for Generic Prompt Condition... Table 8Ð6: Predicting Coherence for Directed and Generic Prompts... Table 8Ð7: Regression Coefficients for Predicting Guidelines Quality for Directed Prompt Condition... Table 8Ð8: Regression Coefficients for Predicting Guidelines Quality for Generic Prompt Condition... Table 8Ð9: Predicting Guidelines Quality for Directed and Generic Prompts... Table 8Ð10: Mean Differences in Coherence for Autonomy Groups...

9 Table 8Ð11: Mean Differences in Coherence for Principle-Citing Groups... Table 8Ð12: Mean Differences in Guidelines Quality for Principle-Citing Groups... Table 8Ð13: Mean Differences in Overall Critique Quality for Elaboration Groups... Table 8Ð14: Mean Values of Overall Critique Quality for Elaboration Groups by Condition... Table 8Ð15: Mean Values of Guidelines Quality for Uncodable Groups by Condition... Table 8Ð16: Mean Values of Overall Critique Quality for Non-Reflective Groups by Condition... Table OÐ1: Boys' and Girls' Autonomy Beliefs, by Pre-test Scores... Table OÐ2: Boys' and Girls' Process Beliefs, by Pre-test Scores... Table OÐ3: Mean Values of Guidelines Quality for Principle-Citing Groups by Condition... Table OÐ4: Mean Values of Critique Quality for Uncodable Groups by Condition...

10 Acknowledgments I would like to thank the eighth grade students who participated in this research. Without them, this dissertation would not exist. Their teacher, Doug Kirkpatrick, has helped me in more ways than I could ever nameñespecially by being such a wonderful teacher and role model. I am grateful to my advisor, Marcia Linn, whose insight and experience have been invaluable throughout my years of working with her. Because of these two exceptional mentors, I leave graduate school far more able to help change the field of education. I also want to thank my other committee members, Barbara White and Mike Clancy, for their interest, guidance, and support over the years I've been at Berkeley. In different ways, they both have helped me remember to notice both the forest and the trees. My colleague and friend Philip Bell has helped me with this research at every step of the way, and for his help in thinking about this work I am more than thankful. I also appreciate the ongoing help of my other dissertation group colleagues. In particular, thank you to Dawn Rickey for providing excellent suggestions at all levels of this dissertation and the research that has preceded it. I've also been lucky to be a part of three different research groups over the last six years, and I appreciate the ways in which the members of these groups have helped me develop my ideas. Last, I want to thank my family and nearlyfamily. To Garrett Scott and the rest of the book/camping club, to Tom and Betsy Davis, and to Dick and Kathy Davis: Thank you all for the support you've given me while I've worked on this research. You've helped me financially, emotionally, psychologically, intellectually, and physically, and I can't imagine doing this without you.

11 Scaffolding Students' Reflection for Science Learning Copyright 1998 by Elizabeth Anna Davis

12 Abstract Scaffolding Students' Reflection for Science Learning by Elizabeth Anna Davis Doctor of Philosophy in Education University of California, Berkeley Professor Marcia C. Linn, Chair Research in recent decades has emphasized the importance of reflection for students learning science, but educators have not reached consensus on the most effective ways to promote reflection, nor has a mechanism explaining the effects of reflection been accepted. Furthermore, many have put forth technology as a vehicle for improving student learning, yet others discount its ability to facilitate real reflection. This research determines whether reflection prompts promote knowledge integration for students working on science projects and what level of prompt specificity best supports students in that endeavor. The Knowledge Integration Environment (KIE) affords investigation of computer-delivered prompts for students completing complex projects. This research takes place in the context of the KIE software and curriculum as used in an eighth grade physical science class. Pilot research on prompts indicated that focusing students on reflection significantly increased knowledge integration. A basic question unanswered by the pilot research was: As students work on projects like those used in KIE, do they merely need to be prompted to reflect, or do they need guidance in determining what to reflect about? The prompts contrasted in this research differ in their specificity. Some students received directed prompts aimed at fostering planning and self-monitoring, while others received generic 'stop and think' prompts. The investigations describe the gross effects of reflection prompts, then attempt to identify a mechanism behind those effects through characterizing the kinds of reflection they elicit and the beliefs about science and learning science of individuals using the prompts. I argue that by engaging in reflection, students identify weaknesses in their knowledge and then are more ready and able to link and distinguish their ideas. Generic prompts are more effective than directed prompts at engaging students in these knowledge integration processes. Autonomous students benefit most from generic prompts for reflection.

13 This research contributes to teaching practice, technology design, and the educational and cognitive research literature. The success of generic prompts, in particular, indicates that instructional designers should concentrate on building learning environments that provide opportunities for students to reflect, and allow students to take responsibility for directing their own reflection autonomously. Chair

14 Chapter 1: Introduction Introduction and Design This research is designed to determine (a) if reflection prompts promote knowledge integration for students working on science projects and (b) what level of prompt specificity best supports students in that endeavor. Here, I use the term "reflection" to refer to both metacognition and sense-making. For example, reflection can focus on one's own thinking or goals, or on content itself. I look at reflection facilitated by sentence-starter prompts that explicitly call for this special kind of thinking. For example, a reflection prompt might say, "In thinking about how it all fits together, we're confused abouté"; a typical response in a science class studying energy conversion would be, "why black gets hotter than white." My goals for this research are twofold. Not only do I hope to develop ways to encourage knowledge integration through facilitating reflection, but I also hope to improve educational practice and inform the design of instruction through a synergy of technology and pedagogy. This work is situated in the perspective of knowledge integration (Linn, 1995; Linn & Eylon, 1996). When I say "knowledge integration," I mean, at its most basic level, the process of linking scientific ideas together to develop a robust, coherent, conceptual understanding. Knowledge integration represents a view of how students learn. Specifically, in the current work, I am interested in how students learn science. However, the knowledge integration perspective can be applied to learning in any domain. The knowledge integration research is situated in the larger literature on conceptual change and has evolved over the past decade. Reflection provides one method for fostering knowledge integration by helping students to expand their repertoire of ideas, differentiate among them, and make connections between them. With reflection prompts, I hope to foster the knowledge integration processes that happen naturally for some students and help students learn to engage in these processes autonomously over time. Most students need many opportunities to reflect to build cohesive, coherent accounts of new material. The Knowledge Integration Environment (KIE) affords investigation of computer-delivered prompts for students completing complex projects. This research takes place in the context of the KIE software and curriculum, in an eighth grade physical science classroom using the Computer as Learning Partner (CLP) software and curriculum, as well. KIE projects scaffold students 1

15 in making sense of complex information from the World Wide Web. Students think critically about "evidence" and use it to build arguments. In more traditional science classes, reflection is less important because students can succeed without it; in the CLP/KIE classroom, though, reflection plays a paramount role, because it helps students set goals and monitor (and improve) their understanding. (CLP and KIE will be described in greater detail in Chapter 2.) This research on reflection prompts has grown out of the extensive history of the idea of reflection. Today's educational researchers generally accept the need for reflection in students' learning, although it is not always incorporated into instructional practice. My work builds on research showing that when students reflect on their ideas they generally produce better products. I postulate that prompting students to reflect can set this process in motion. The process of reflecting on ideas may help students identify weaknesses with their current understanding and thus motivate them to revisit, test, and reformulate the links and connections among their ideas, leading to more coherent, robust, and integrated understanding. This research describes effects of prompts, identifying features of particularly promising ones. I ask, Do students merely need to be prompted to reflect, or do they need guidance in determining what to reflect about? This research also delineates mechanisms that explain how the prompts facilitate reflection, and how reflection, in turn, is linked to knowledge integration. I ask, Do all focuses of reflection lead to the same results? I also investigate the role of individual students' beliefs about science and learning science in their reflection and learning. I ask, Do all students benefit equally from the same type of prompts for reflection? To investigate these questions I contrast two types of reflection prompts. The first type, called generic prompts, represents a view that asking students to "stop and think" will encourage reflection. The second type, called directed prompts, assumes that a generic request for reflection is insufficient, and that students should instead be provided with hints indicating potentially productive directions for their reflection. An example of a generic prompt is, "Right now we're thinkingé." An example of a directed prompt is, "To do a good job on this project, we need toé." The rationale for each of these types of prompts is given in Chapter 2. Knowledge Integration I view science learning as a process of integrating ideas (disessa, 1988; Linn & Eylon, 1996). To integrate ideas, students add information, reorganize information, promote some ideas, and demote other ideas. The ideas students 2

16 bring to science class get linked to new ideas, combined with each other, and reorganized. Through the process of knowledge integration, students develop a coherent, robust understanding of science concepts. In this process, students expand their repertoire of ideas, discriminate between ideas, and reorganize the links among them (Linn & Eylon, 1996). Expansion of the repertoire of ideas is necessary but not sufficient (Hsi, 1997). Some students are content to accept any idea presented, without consideration of whether that idea makes sense on its own or of how it fits with other ideas. These students are less successful at developing an understanding of scientific phenomena than are students who reflect on new ideas, working to understand each concept; distinguish between ideas; determine places where links can be made among the ideas, thus improving their knowledge integration; and identify weaknesses in their current knowledge. Let us consider each of these processes in turn. First, when students expand their repertoire of ideas, it is more useful if they work to understand a new concept before they add it. For example, if a student is exposed to a new idea (say, that black absorbs light), the student must develop an understanding of that concept before it can be linked appropriately to other ideas. Students must distinguish between ideas. For example, students may start out considering the ideas "black attracts heat" and "black absorbs light" as equivalent. Later, they may come to recognize the importance of the nuances of scientific language, and may distinguish between attracting and absorbing. They may now consider "black absorbs heat" and "black absorbs light" as equivalent, though. Eventually, they may distinguish between the concepts (and the words) "light" and "heat," and may come to believe that black objects actually absorb light. Students also need to make connections among ideas. For example, they may believe that black absorbs light. They might also know that black objects get hotter than white objects. Connecting these ideas, though, can be difficult. A useful connection in this case would be, "Absorbed light is converted into heat energy." Through a connection like this students can come to have a more robust understanding of scientific phenomena. To reach that point, though, they must identify a weakness in their knowledge. Many students are perfectly content with "knowing" that a phenomenon is "true." To them, knowing that black objects get hotter is sufficient. However, in the CLP/KIE classroom, students are required to make scientific explanations; descriptions of phenomena are not enough. Often through one-on-one conversations with students, we can help them identify 3

17 where their knowledge has weaknessesñthat is, where links, distinctions, or reorganizations could be made or where new ideas could be added. The weakness might simply be a place where a new idea can be added to the repertoire. For example, a student may notice for the first time that one gets warmer wearing black during the day, but experiences no temperature change at night. They might add the idea "Black objects do not get hotter when they are in the dark." Or, the student might identify where ideas should be distinguished from one another, as in the distinction between light and heat discussed above. The weakness might alternatively be a place where a link can be made between two ideas. A direct link might be made between the ideas that black T-shirts and black asphalt both get hot on a summer day, for example. A more sophisticated link might connect one or both of those ideas with the knowledge that dark colored objects are harder to see at night. Through identifying where links like these could be made, students can develop new models or explanations for phenomena. For example, the student might link the two principles "black absorbs light" and "absorbed light is converted to heat energy" or the student might link the idea "black gets hotter" and the principle "black absorbs light." Identifying where these weaknesses are in the current knowledge will help the student develop a coherent, integrated understanding of energy conversion by illuminating the need to engage in more thought. The research presented in this dissertation will investigate ways in which reflection can help students engage in these knowledge integration processes. Knowledge Integration and Conceptual Change How does knowledge integration relate to conceptual change? Through the processes of knowledge integrationñdistinguishing between ideas, linking ideas, and identifying weaknesses in one's knowledgeñlearners revise their understanding of scientific concepts and develop a coherent, integrated understanding. Knowledge integration, then, is a mechanism of conceptual change. Students often have non-normative understandings of science concepts. These conceptions are generally grounded in their phenomenological experience (Inhelder & Piaget, 1958). For example, we have all had the experience of sitting down on a metal chair and having it feel cold. Students may believe that the chair is, in fact, colder than, say, a wooden table, even if the two objects have been in the same surroundings for several years. The literature would suggest many different ways in which students might come to understand that "objects in the same surroundings are at thermal equilibrium," and that "some objects feel colder than others in part because of 4

18 differences in the rate of heat flow through the materials." Accompanying these different views of how conceptual change happens cognitively are different instructional approaches to achieve conceptual change. Here I contrast the theoretical and instructional implications of many of these views with the implications of the knowledge integration view. Replacement of Misconceptions Some researchers would consider students' non-normative beliefs as "misconceptions" to be replaced or eradicated. Much of traditional instruction takes the approach that telling the students more scientifically normative ideasñpresenting them with an explicitly conflicting "correct" ideañwould help them develop an improved understanding (McCloskey, 1983). Students might be "induced" to give up their intuitive beliefs and adopt instead the scientifically accepted ideas. In the thermal equilibrium example described above, a misconceptions approach would view the "problem" as lying with students' understanding of energy at a fairly global level, and instruction might involve telling students that objects in the same surroundings are at thermal equilibrium. As a result of instruction like this, students might adopt the scientific explanation, at least temporarilyñbut their understanding would not be likely to be robust, because they would not link the new information with their existing ideas. Conflict Findings regarding conflict as a mechanism for conceptual change have been inconclusive, though Chan and colleagues identify maximal conflict as most successful for those students who have a propensity toward knowledgebuilding, as opposed to assimilation (see Chan, Burtis, & Bereiter, 1997, for a review of this literature, as well). Piaget maintained that children learn when their understanding becomes disequilibriatedñthough he did not recommend conflict as a method for encouraging learning (Piaget, 1952). Constructivist theory holds students' experience-based conceptions as something to build upon, rather than something to replace. In this view, learners should struggle with ideas and build upon their existing knowledge. (See Smith, disessa, and Roschelle [1993] for a discussion of these issues.) Dissatisfaction with Ideas Strike and Posner (1992) claim that for conceptual change to take place, students must first be dissatisfied with their current idea. They must also find the new idea intelligible; under this constructivist approach (unlike, perhaps, 5

19 the misconceptions approach), students will not adopt a new idea if they do not understand it. But furthermore, they must see the new idea as both plausible and as a fruitful replacement for their old idea (Strike & Posner, 1992). In the knowledge integration approach, on the other hand, the old idea would not be replaced but instead would be applied less and less frequently. White and Gunstone (1989) extend Strike and Posner's view, adding explicit, sustained reflection to the necessary conditions for conceptual change. How would this very rational approach play out in the thermal equilibrium example? Students would first need to decide, for some reason, that materials cannot in fact be "naturally cold." They might, for example, be asked to think of situations in which that explanation is inadequate. They would need at the same time to have under consideration some alternative explanationñfor example, that rate of heat flow might be an explanation. But in order to determine that rate of heat flow is a plausible and fruitful explanation, they must first make sense of the concept of rate of heat flow itself; it must be an intelligible idea to them. They might make predictions about the rate of heat flow through different materials, doing a laboratory experiment where they attach balls of wax to the ends of bars of different materials, and then heat the bars. Through experiments like this they might come to consider rate of heat flow as intelligible; the next step would be to identify it as a plausible and fruitful alternative. Rate of heat flow might be a plausible explanation if students understand that humans are heat sources and that a bench might feel cold if heat is flowing quickly from the body into the material. And, the idea might be adopted as fruitful when students come to recognize both the intractability of their intuitive ideas and the appeal of new ones. Theory Change Kuhn's (1970) ground-breaking work on the development of new knowledge in science provides an over-arching analogy of a "revolutionary" rather than "evolutionary" stance. Carey (1988) postulates that students have theories about science and that those theories need to be revised. Unlike the misconceptions camp, though, which anticipates theories being replaced through didactic instruction, Carey argues that theories are changed through a series of differentiations and coalescences. Instructionally, this would mean that students would be engaged in activities designed to help them conceptually understand the topic at hand. For example, Carey and her colleagues have engaged students in a several-week inquiry-based curriculum designed to help students revise their ideas about the nature of science, in the context of thinking about whether yeast is alive (Carey, Evans, Honda, Jay, & Unger, 1989). 6

20 Analogously, we could imagine engaging students in a series of lab activities to help them recognize that objects in the same surround are at the same temperature. They might also do labs designed to help them understand rate of heat flow as a possible explanation, and to identify themselves as heat sources. The activities would not necessarily differ very much from those used in the CLP/KIE classroom. However, we view students as linking ideas rather than revising theories (disessa, 1988; Linn, 1995). Driver and her colleagues argued against the theory revision stance toward conceptual change, claiming that students' knowledge is too tacit and situated to constitute theories (Driver, Asoko, Leach, Mortimer, & Scott, 1994). Category Change Chi and her colleagues postulate that students' ontological categories (not their ideas or theories themselves) need improvement (Chi, 1992, 1993). Chi holds that people have ontological categories into which they put ideas and concepts. Sometimes, learning is relatively easy because it does not require a shift in ontological category. An example would be when young children reorganize their knowledge in recognition that humans are animals rather than representing a separate form of life. Chi would not consider this kind of learning to be conceptual change. Other times, though, students need either to move an idea into a different existing category or to develop a new category into which to put the idea. An example of a shift in ontological category like this is given in Chi and Slotta's work in electricity. They found that students could learn about the topic of electricity much more easily if they were first trained in the category of "constraint-based interactions" (Slotta & Chi, 1996). In learning about heat, as with electricity, students are likely to have a "material substance" view that might cause difficulty with learning. From this perspective, in the thermal equilibrium example students might be trained in constraint-based interactions to help them move away from thinking of heat as a substance and toward recognizing equilibrium as one of many examples of an "equilibration process." Isomorphic problems might be used in this training. However, our work in CLP indicates that some ideas like this are actually strong foundations for students to build on, even when they are not scientifically normative (Linn & Muilenburg, 1996). Restructuring Ideas All of these represent potentially useful ways of thinking about conceptual change. My work, however, is based in the knowledge integration view, in which students need to distinguish between ideas, link ideas, restructure ideas, and identify weaknesses in their knowledge to develop a truly robust, 7

21 coherent, integrated understanding of science concepts. In knowledge integration, students' initial conceptions are viewed as productive ideas to be built upon, rather than misconceptions to be eradicated. We address students' existing conceptions through scaffolding them. Sometimes ideas need to be distinguished from one another, and sometimes new links need to be made between existing ideas. New ideas also, of course, need to be added to the repertoire. Instructionally, students should be exposed to a repertoire of alternative models (Linn, disessa, Pea, & Songer, 1994). This gives them access to intermediate (often qualitative) models that could more easily relate to their alternative conceptions, which might then be refined and evolved into more sophisticated models (White, 1993). Sometimes old, nonnormative ideas fall out of use when new ideas are added to the repertoire and ideas are linked in new ways; this does not mean that the old, less fruitful or satisfying ideas are gone from the students' repertoire. Often even expert scientists rely on non-normative ideas in certain contexts (Linn & Muilenburg, 1996). In the knowledge integration approach, learners are not viewed as having theories about science concepts nor is the learning process viewed as one in which categorical shifts are at the basis of learning. Students' understandings are instead viewed as webs of loosely- and tightly-linked ideas, which undergo a process of integration and restructuring (Linn et al., 1994; Smith et al., 1993). This dissertation will investigate how reflection can help foster knowledge integration to help students develop a more coherent understanding of science ideas. Rationale for Reflection Prompts How does reflection help students integrate their knowledge? Recall that I use the term "reflection" to refer to something with both metacognitive and sense-making components. Metacognition has been defined as "knowledge of cognition" and as "regulation of cognition" (Brown, Bransford, Ferrara, & Campione, 1983). I break metacognition into three differentiable pieces having to do with both self-monitoring and self-regulation: reflection on one's thinking, learning goals, and behavioral goals. For students, one's learning goals may concern the product of instruction while one's behavioral goals may concern the activities inherent in that instructional contextñthat is, the process of participating in the instruction. (Students who reflect on their behavior might plan to "work hard" or "listen to the teacher," for example, while those who reflect on their learning goals might plan to "do a good report." Students who instead focus on their thinking might anticipate 8

22 identifying "how this relates to the lab we just did." Of course, these focuses generally occur in combinations, rather than in isolation.) Experts also reflect on their content knowledge and engage in sense-making. (Students reflecting on content might wonder why black absorbs light, for example; they might then work to understand this question.) People who reflect on content integrate their knowledge and are able to apply it. However, metacognitive activity is also necessary for knowledge integration, since learners must identify weaknesses in their current knowledge before they begin to distinguish between and link ideas. Reflection on cognition and reflection on content are closely tied. My hypothesis is that reflection, used generally, can help students undergo the processes of distinguishing between ideas, making links between them, and identifying weaknesses in their current knowledge. Reflecting on one's thinking, one's learning goals, and (to some extent) one's behavioral goals primes students for engaging in these processes; some reflection is more productive than other reflection because it is more likely to foster knowledge integration. A student who ponders his or her level of understanding, who plans for the day's work, and who considers ways in which the current ideas are or are not well-understood will be more apt to engage in knowledge integration than a student who merely forges full-speed ahead without engaging in these metacognitive activities or who only reflects on how they behave, rather than how they think. In particular, students who recognize weaknesses in their current understanding are more apt to engage in the other knowledge integration processes of linking and distinguishing ideas. But, being primed for knowledge integration is not the same thing as engaging in it. When students take the opportunity to reflect on their understanding of science concepts and ideas, they identify places where new ideas could be added and links and distinctions should be made. Sentence-starter reflection prompts in KIE are used to promote explicitly this metacognitive and sense-making reflection. An interesting feature of reflection prompts is that students can interpret them in any way they wish. Because of this characteristic, reflection prompts are intrinsically metacognitive and only potentially do they provide a locus for sense-making: Students must interpret them in a certain way for them to be used as a sensemaking opportunity. Reflection prompts represent the only explicit call for metacognition that students receive in KIE projects, though sense-making is an expectation throughout the projects. A more specific rationale for the reflection prompts contrasted in this research will be given in Chapter 2. 9

23 The Scaffolded Knowledge Integration Framework and Prompting KIE projects require students to engage in sustained reasoning. Our goal with KIE projects is to help students develop an improved conceptual understanding and an improved ability to think critically about evidence and use it effectively. Proper scaffolds are necessary to help students succeed at these difficult tasks. Reflection prompts represent one of these scaffolds. Reflection prompts fit into the scaffolded knowledge integration framework, which frames the instruction for CLP and KIE. Scaffolded knowledge integration is the instructional approach used to support knowledge integration: Students are scaffolded, or supported, as they make links among their ideas. The framework has grown out of 12 years of design studies and involves four elements (Linn, 1995). First, instruction should "make thinking visible" to students by illustrating how links and connections are made. Teachers and students reasoning about scientific phenomena need to reveal their own thinking to themselves and their peers. Second, instruction should identify models for scientific phenomena that make sense to students so they can connect new information to existing knowledge and to problems that are both familiar and relevant. Third, instruction should provide social supports so all students learn new links and connections for their ideas from their peers. Finally, students should be encouraged to become autonomous learners so they can regularly revisit their ideas and continue to engage in knowledge integration. These four elements jointly promote knowledge integration. Although reflection prompts contribute to each of the tenets of the scaffolded knowledge integration framework, they particularly encourage autonomy by providing an explicit place for reflection. By learning how to engage in reflectionñrather than just memorization, as is often encouraged by their other experiences with scienceñstudents may begin to take responsibility for their own learning. Reflection prompts scaffold this process. We also hope they help students to develop the propensity to continue linking ideas and evaluating views autonomously. To promote autonomy, I designed prompts to encourage reflection with the idea that regularly engaging in reflection would illustrate the advantages of reflection and lead to autonomous reflection in the future. Prompts may help students develop the disposition toward regular reflection (cf. Perkins, Jay, & Tishman, 1993; Resnick, 1987). Reflection prompts also help to make thinking visible. Prompts can model some of experts' thinking practices. Directed prompts, in particular, model the planning and monitoring in which experts engage. Reflection prompts also help students make their own thinking visible by providing an explicit place for reflection. As with thinking aloud (e.g., Bereiter & Bird, 1985; Collins & 10

24 Smith, 1982; Inhelder & Piaget, 1958; see Kucan & Beck, 1997, for a review of this literature) and self-explaining (Bielaczyc, Pirolli, & Brown, 1995; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, deleeuw, Chiu, & LaVancher, 1994; Webb, 1983), prompts make explicit students' own thinking. However, written prompt responses make their thinking truly visibleñto them, to their teachers, and to researchersñrather than being spoken without record. Hypotheses and Results In this research, I contrast two conditions. During a single semester, three of the KIE/CLP eighth grade physical science class periods received directed prompts, and three received generic prompts as they worked on a KIE critique project. Having reviewed the theoretical framework that provides the foundation for this research, we turn now to a rationale for this design, and an overview of the kinds of outcome measures investigated. I then discuss the predicted outcomes for each of the major investigations and briefly outline the actual findings. First, what are the broad implications of this work? This research contributes to both the educational and cognitive research literature. By identifying the success rates of generic and directed prompts, we can make informed decisions about how best to help students succeed in science. We will learn more about facilitating reflection and about facilitating knowledge integration. Furthermore, by investigating the cognitive mechanisms explaining how reflection prompts facilitate knowledge integration, we continue to refine our understanding of how reflection and learning are related. And finally, by investigating the relationships among students' beliefs, their reactions to different instructional approaches (here, prompts), and their ability to integrate their knowledge, we are better able to improve learning for students with particular characteristics. Understanding the effects of prompts and the mechanisms behind those effects will help us support students individually and collectively as we develop curriculum materials and technologies for learning. Investigating Prompts for Reflection Technology affords us the opportunity to offer students prompts tailored to student and instructional characteristics. The pilot research discussed in Chapter 3 indicates that students benefit from prompts encouraging reflection. What remains unclear is whether that benefit is a result of the implicit instruction to reflect on X, or whether the mere act of stopping to reflect is the cause of the benefit. Thus, this dissertation compares prompts that vary in specificity: directed and generic reflection prompts. 11

25 In investigating the effects of directed and generic prompts on student learning, we must first identify some measures of those effects. The measures used in this research include the kinds of cites students make and link to one another (i.e., do they cite principles, labs, experiences, etc. in explaining scientific phenomena?) and the overall quality of their projects themselves. The KIE project investigated in this research is intended to improve both students' conceptual understanding of certain science ideas and their ability to critique scientific evidence and claims. Thus, the measures of project quality include the coherence of their ideas, the quality of their critiques, and their ability to abstract out from the act of critiquing to develop guidelines for critiquing. These measures depend in part on the students' ability to identify weaknesses in ideasñtheir own or those of others. One question this dissertation will address is, which kind of prompt is most helpful at helping students see these weaknesses? One potential benefit of generic prompts is that they are unintrusive. Such prompts are less likely than directed prompts to interrupt productive activities with requests that are potentially irrelevant to the students. We might hypothesize that generic prompts would be best for students who are already likely to succeedñfor example, for students who realize that learning involves the revision of ideas and that they are responsible for seeing where their knowledge is problematic. Directed prompts, on the other hand, are more supportive. They model productive kinds of reflection for students. Some students may need such guidance to help them reflect effectively. Furthermore, the pilot work indicates that prompts focused on selfmonitoringñthe prompts on which directed prompts are basedñare successful at eliciting knowledge integration. It may be the case that students need directed prompts until they realize that reflection is an important part of working on a complex project. After that point, they may benefit just as much from generic prompts. We might hypothesize that directed prompts would in particular be more useful, overall, in helping students develop good critiques; in response to directed prompts students might create plans emphasizing the salient aspects of critiquing. The effects of the two types of prompts on students' work on the project are discussed in Chapters 5 and 8. I find that the effects of prompts propagate through the rest of the project. Specifically, I find that generic prompts elicit more cites of principles and evidence, and more cites in general, when students work on the project itself. Citing principles, in particular, is found to be positively related to developing a coherent, integrated understanding and, to a lesser degree, to being able to critique. Students in the generic prompt condition who cited principles also developed high-quality guidelines for critiquing, and students in the generic prompt condition who cited many ideas overall did better at critiquing. Students who cited few principles 12

26 developed less coherent ideas and did worse at writing guidelines for critiquing. Generic prompts thus appear to elicit a broad range of productive ideas, helping students expand their repertoires of ideas. There were also differences in the conceptual understanding of students in the two conditions. Students who received generic prompts developed a better understanding of the individual ideas of energy conversion and thermal equilibrium more often than did students who received directed prompts. Students who received generic prompts developed a more coherent, integrated understanding of the science overall. I hypothesize that this stems in part from the broader range of ideas elicited by the generic prompts. However, we also see in Chapter 5 that students in the two conditions did not differ in their overall project scores, the quality of the guidelines they develop, or the quality of their critiques. We see that abstracting and critiquing, in particular, are hard for students to do well, though students who were successful at one quality measure tended to be successful at others, as well. Investigating Focus of Reflection Also under investigation is how the focus of the reflection in response to prompts is related to students' success on their science projects. What weaknesses do they detect in their knowledge? What else do they choose to consider? Pilot research identifies distinct differences in students' interpretations of even very directed prompts. For example, when asked to assess their understanding, some students focus on science concepts, while others focus on logistical aspects of the project. The responses to the reflection prompts themselves are investigated here to investigate the nature of knowledge integration. Each response is coded for its degree of elaboration and the focus of its reflection. Students' focuses of reflection are then linked to their success on the project, to identify possible relationships between the focus of reflection and knowledge integration. I investigate differences in how students reflect as a result of the two levels of specificity of prompts. Content, cognitive, and goal-oriented focuses might be reactions to various types of weaknesses; these focuses can all be useful. If a prompt encourages students to think about the "big picture"ñfor example, what their goal is for the project or how they should accomplish that goalñthey may work more productively than if they simply plow ahead. On the other hand, if a prompt encourages students to contemplate their own conceptual understanding, 13

27 they should also benefit. Students who are content-focused may develop more integrated knowledge, while students who are more goal-focused may do a better job of critiquing evidence. If students focus instead on their behavior, they may do unremarkable work. Generic prompts leave the context to the students to provide. One might predict that many students' focus will be on instructional goals since that may be considered a more manageable task. Directed prompts are more likely to point students in one direction or the other, but the prompts leave room for interpretation, as shown by the range of student responses in the pilot work. I hypothesize that certain orientations might be beneficial in response to one type of prompt, but less useful when made in response to the other type, because the benefit of reflection might depend in part on the context in which the reflection takes place. In Chapters 6 and 8 I outline the results of these analyses of students' responses to prompts. I find that elaboration in response to directed prompts may help students develop better critiques. I find that for both types of prompts, students most often focus on specific instructional goals (e.g., identifying criteria for critiquing evidence). I also find that generic prompts were less likely to enable a lack of reflection than were directed prompts, and elicited more reflection on science content. However, while focusing on content in response to directed prompts may improve the coherence of students' ideas seen in their projects, reflection on content in other instances may have negative effects. For example, focusing on content in response to generic prompts may reduce students' performance in all three major measures of project quality: coherence of ideas, overall critique quality, and guidelines quality. Pairs who are cryptic in response to directed prompts tend to have poor performance overall, and those who are non-reflective in response to these prompts (i.e., who say they understand everything) tend to have poor critiques. This finding leads us to the other aspect explaining generic prompts' effectiveness: Students who do not take advantage of the opportunities prompts afford for reflection are less likely to identify weaknesses in their current knowledge, and as a result do not integrate their knowledge as well. Since generic prompts are less likely to elicit this minimalist approach, their effectiveness is improved. Overall, we see that taking specific actions in response to directed prompts can have both strong positive and strong negative effects on students' work on the project, depending on the action. Generic prompts, on the other hand, seem to have more generally positive effects. When are the seemingly productive activities promoted by generic promptsñmost notably, reflecting 14