RUNNING HEAD: REASONING IN SCIENCE CLASSROOM DISCOURSE A FRAMEWORK FOR ANALYZING EVIDENCE-BASED REASONING IN SCIENCE CLASSROOM DISCOURSE

RUNNING HEAD: REASONING IN SCIENCE CLASSROOM DISCOURSE A FRAMEWORK FOR ANALYZING EVIDENCE-BASED REASONING IN SCIENCE CLASSROOM DISCOURSE Erin Marie Furtak University of Colorado at Boulder Ilonca Hardy University of Frankfurt Christina Beinbrech University of Münster Jonathan T. Shemwell Richard J. Shavelson Stanford University Paper Presented at the European Association for Research on Learning and Instruction, August 25-29, 2009, Amsterdam, Netherlands

REASONING IN SCIENCE CLASSROOM DISCOURSE 2 Abstract Although students ability to participate in scientific argumentation is considered a major objective of science education, analyses of classroom discourse have revealed that unsupported conjectures are prevalent in elementary school (Newton & Newton, 2000) and even in secondary school (e.g., Osborne, Erduran, & Simon, 2004). This paper combines and extends prior work on student-reasoning frameworks to develop a new analytic tool for reasoning in science classroom discourse. The instrument, Evidence-Based Reasoning in Science Classroom Discourse, is intended to provide a means for measuring the quality of student reasoning in whole-class discussions, capturing teachers and students co-constructed reasoning about scientific phenomena. This paper describes the features of the instrument, illustrates how it can be applied to an instance of classroom reasoning, discusses its strengths and weaknesses, and makes recommendations for future analyses. Its significance lies in presenting a new analytic approach to assessing students ability to reason in classroom discourse.

REASONING IN SCIENCE CLASSROOM DISCOURSE 3 Introduction While assessment is commonly conceived as a paper-and-pencil activity the teacher asks the students to do independently, the informal interactions teachers have with individual students or, for that matter, a whole class, also can serve as opportunities to conduct assessment on-the-fly to determine the status of student thinking (Author, 2008a). In fact, whole-class discussions designed to elicit student thinking so that the teacher may use that information to take instructional action have been identified as an important formative assessment tool (Duschl & Gitomer, 1997). Just as these assessment conversations may serve the purpose of having students share their thinking as it develops in a conceptual domain (Author, 2008b), whole-class conversations can also serve as opportunities for the teacher to elicit and develop students ability to engage in scientific processes. Recent science education reforms have emphasized student participation in the process of scientific inquiry, whereby students engage in what the National Research Council (2001) called the Five Essential Features of Inquiry. Primary among these features is the collection and analysis of evidence in support of developing and communicating scientific explanations. Therefore assessment conversations oriented around evidence have the potential to serve multiple concurrent purposes; not only can they engage students in the process of evidence-based reasoning and the social facet of science (Duschl, 2003) they can also make public students reasoning processes so that the students may hear and consider alternative explanations to their own, and the teacher may take instructional action to move students toward more sophisticated scientific communication. However, previous analyses of classroom discourse have revealed that unsupported student conjectures are prevalent, especially in elementary school (Newton & Newton, 2000).

REASONING IN SCIENCE CLASSROOM DISCOURSE 4 Even in secondary school, establishing a classroom culture of scientific reasoning seems difficult to achieve (e.g., Osborne, Erduran, & Simon, 2004). Several studies have explored how the process of student reasoning might be supported in science classrooms, and have contributed a number of analytic approaches to help identify high-quality reasoning when it occurs. Some of these approaches build upon Toulmin s (1958/2003) The Uses of Argument, and others make distinctions based on the quality of reasoning (Carey, Evans, Honda, Jay, & Unger, 1989; Driver et al., 1994; Kawasaki, Herrenkohl, & Yeary, 2004; Tytler & Peterson, 2005); however, none integrate methods for analyzing the contribution of the teacher to supporting high-quality reasoning. Based on these prior studies, we developed an instrument that combines Toulmin s elements of reasoning with the assessment of different qualities of evidence-based reasoning, and the degree of teacher support for constructing an argument. This instrument is an adaptation of the Evidence-Based Reasoning Framework (Paper 2, this issue) for the analysis of science classroom discourse. It is intended to provide a means for assessing the quality of scientific reasoning in whole-class discussions, and provides an analytic framework for examining teachers and students co-constructed reasoning about scientific phenomena. Prior Analyses of Evidence-Based Reasoning In Science Classroom Discourse Toulmin s (1958) framework for the analysis of arguments has served as a basis for many instruments for the analysis of classroom discourse in science (e.g., Jimenez-Alexandre, Rodriguez, & Duschl, 2000; McNeill, Lizotte, Krajcik, & Marx, 2006; Simon, Erduran, & Osborne, 2006). Essentially, Toulmin s framework suggests that an argument consists of a claim and supporting evidence, or backing. Many studies have applied Toulmin s framework, with a

REASONING IN SCIENCE CLASSROOM DISCOURSE 5 variety of adaptations and at different grain sizes (e.g., making distinctions between claims and warrants), as an analytic tool for determining the quality of students reasoning in writing (e.g. Kelly & Bazerman, 2003; McNeill et al., 2006; Sandoval, 2003; Sandoval & Millwood, 2005). Discourse in science classrooms, however, is a different kind of communication, involving multiple speakers and particular ways of knowing (Edwards & Mercer, 1987; Lemke, 1990). Therefore, analyses of classroom discourse have necessitated different kinds of analytic frameworks. Eliciting and supporting student contributions involves intervention and support from the teacher; multiple student speakers may develop the same idea; and systematic reasoning structures become more challenging to track. For these reasons, the number of studies that have attempted to adapt Toulmin s framework to classroom discourse is much smaller. Russell (1983) explored how teacher questions helped students develop arguments on the basis of scientific evidence. Taking whole arguments as a unit of analysis, Russell developed a framework that combined research on attitudes toward authority with Toulmin s argument structure. Russell concluded that the manner in which teachers use questions could serve to reinforce traditional authority structures and were geared toward getting students to provide the answers teachers were looking for in the arguments. Felton and Kuhn (2001) developed a coding system to analyze argumentative dialogues, albeit in the domain of capital punishment instead of science. Their framework identified two forms of development in argumentative discourse; first, skill in meeting an activity s objectives by directing the course of an argument; and second, skill in refining the goals being pursued in the argumentation. Codes for transactive questions, transactive statements, and nontransactive statements were applied to individual speaking turns; through this system, the authors were able to identify counter-arguments, rebuttals, and patterns in these and the other codes during

REASONING IN SCIENCE CLASSROOM DISCOURSE 6 dialogues. The authors concluded that adults argued more strategically than teens, whereas teens made use of the same strategies in disagreement and agreement situations. Exploring the Quality of Student Reasoning in Classroom Discussions Another set of studies took a higher-level approach with coding systems or analytic frameworks that explored students` understanding of the nature of science, addressing the relationship between theory and evidence. In coding clinical interviews completed by students after participating in a unit focusing on the formulation and testing of theories, Carey et al. (1989) define three levels of understanding of the nature of science. The first level involves no clear distinction between theory and evidence; that is, collecting facts and seeing if something might work. The second level requires some distinction between theory and evidence; namely, that science is seen as a search for evidence. The third level makes a clear distinction between theory and evidence, where the cyclic, cumulative nature of science is acknowledged, with its goal to construct explanations of the natural world. In this way, Carey et al. (1989) were interested in meta-knowledge, or knowledge about the scientific endeavor. Other studies have used similar distinctions to obtain a measurement of the quality of scientific reasoning about science concepts based on students` ability to differentiate between theory and evidence used as backing. Tytler & Peterson (2005) and Driver et al. (1996) discern among three different levels of epistemological reasoning for student explanations of science phenomena. The first level involves reasoning from phenomena, where explanations of events are equivalent to observing the event itself. The second level, relation-based reasoning, involves generalizations based on causal or correlational relationships. The third level, model-based reasoning, involves the evaluation of theories or models on the basis of evidence. Kawasaki, Herrenkohl, and Yeary (2004) applied this framework to whole-class discussions and small-

REASONING IN SCIENCE CLASSROOM DISCOURSE 7 group interactions, and determined that by making norms of scientific argumentation explicit with these students, students participated more in classroom discussions and became more involved in the process of developing shared meaning and applying the negotiated concepts to various situations. In an analysis of a whole-class discussion, Jimenez-Alexandre et al. (2000) distinguished between doing the lesson, or engaging the ritual activities and procedures of school science, and doing science, a more principled ways of engaging in scientific dialogue and argumentation. Instances of talking science were further coded according to Toulmin s elements of argument (argumentative operations) as well as epistemic operations, including induction, deduction, and causality. Jimenez-Alexandre et al. found that a large part of interactions in the discussion could be classified as doing the lesson, but that in instances of doing science, students developed a variety of arguments. The Role of Teachers in Supporting Scientific Argumentation An important feature and possible influence on reasoning in science classrooms is the extent to which the teacher supports or actually participates in the process of student argumentation. While the aforementioned studies have explored the quality of reasoning in small group and whole-classroom talk, they have not explicitly explored the extent to which the teacher takes steps to activate and advance student argumentation and evidence-supported reasoning. The support teachers provide during whole-class discussions can also be conceived as scaffolding student reasoning. Bruner (1960) defined scaffolding as a way that assures that only those parts of the task within the child s reach are left unresolved, and knowing what elements of a solution the child will recognize though he cannot yet perform them (p. xiv). Through the

REASONING IN SCIENCE CLASSROOM DISCOURSE 8 strategies teachers use during discussions, teachers can coordinate a zone of social learning around students (e.g., Lave & Wenger, 1991; Vygotsky, 1978), which has at its center the students and what they are able to collectively establish through discussion, and on the outside the assistance the teacher provides. Therefore, identifying the manner in which teachers are contributing to the reasoning being done by students can provide a measure of the amount of scaffolding being provided. If students are engaging in high-quality reasoning only when involved in IRE-type exchanges with the teacher (Mehan, 1979), then it is difficult to judge students ability to reason independently from evidence. In contrast, if teachers are asking openended questions that are true requests for information from students (Cazden, 2001), and students are able to continue reasoning on their own without intervention from the teacher, then one may conclude that students ability to reason has developed beyond the point at which it is heavily reliant upon the teacher s intervention. Duschl and Gitomer (1997) suggested that instructional dialogues, which they called assessment conversations, can be especially designed to engage students in discussion of diverse student-generated ideas focused upon evidence and scientific ways of knowing. In the Science Education through Portfolio Instruction and Assessment (SEPIA) project, these assessment conversations were used to help teachers provide scaffolding and support for students construction of meaning by carefully selecting learning experiences, activities, questions, and other elements of instruction. A central element of the assessment conversation is a three-part process that involves the teacher receiving student ideas through writing, drawing, orally sharing, so that students can show the teacher and other students what they know. The second step involves the teacher recognizing students ideas through public discussion, and the third has the teacher using ideas to reach a consensus in the classroom by asking student to reason on the basis

REASONING IN SCIENCE CLASSROOM DISCOURSE 9 of evidence. Focusing the conversation around evidence makes this example of formative assessment specific to scientific inquiry. Duschl and Gitomer suggested that teachers should focus less on tasks and activities and more upon the reasoning processes and underlying conceptual structures of science. In a qualitative study of his own teaching, Yerrick (2000) followed general guidelines to help students formulate arguments by basing his instruction on students own questions. Following activities students discussed their findings with the whole class; the class usually ended with students agreeing on an explanation or returning to their experiments to gather more data. In comparing interviews at the beginning and end of the school year, Yerrick found that students sophistication in formulating scientific arguments in response to the questions investigated throughout the school year did improve. However, to what extent these questions or prompts actually lead to an increase in the quality of students scientific argumentation and evidence-based reasoning has to date not been empirically tested in a comparative study. More recent studies have found that varying levels of teacher intervention may help students to develop conceptual understanding (Author, 2006; Scott, 2004; Scott, Mortimer, & Aguiar, 2006), suggesting that ongoing classroom discussions may involve different levels of support from the teacher. For example, the emphasis on empirical support of arguments increases has been suggested as a vehicle to promote reasoning about scientific phenomena (see Tytler & Peterson, 2005). Toward a New Framework for the Analysis of Evidence in Science Classroom Discourse To serve our own purposes of tracking the quality and quantity of student reasoning in assessment-oriented classroom conversations, we have combined the approaches of argument structure based on Toulmin s distinctions with the three levels of reasoning developed by Tytler

REASONING IN SCIENCE CLASSROOM DISCOURSE 10 and Peterson (2005) to arrive at a classification of instructional discourse which can be rated in terms of different levels of argumentative structure, moving from unsupported claims to claims supported with single data, relational evidence, or models and rules. This paper adapts the conceptual model proposed in this issue (Paper 2, this issue) into a tool intended to analyze science classroom discourse. The extent to which particular student claims about certain premises are supported (or not) by data, evidence, or rules, is analyzed, as is the extent to which the teachers guiding the discussion supports and participates in that process of reasoning. The following section will describe features of the instrument, describe how it is to be applied to instances of classroom discourse, and illustrate how it is applied to a short excerpt of science classroom talk. Evidence-Based Reasoning in Science Classroom Discourse: An Overview of the Instrument The analytic framework presented in this paper was developed in collaboration between German and American researchers who had collected elementary and middle school data on students whole-class discussions during inquiry-based units about the concept of sinking and floating. This work allowed us to explore the use of evidence across multiple grade levels in similar content areas, as well as teacher practices in different educational systems. The Evidence-Based Reasoning in Science Classroom Discourse Framework (EBR- Discourse) captures teachers and students co-constructed reasoning about science phenomena and quality of the backing for those claims. On a continuum of reasoning, the most sophisticated science discourse is conceptualized to consist of claims about science phenomena that are supported by a generalized statement about relationships between properties (a rule). In addition to this statement of a rule, (empirical) backing such as reference to observations (data) or

REASONING IN SCIENCE CLASSROOM DISCOURSE 11 summaries of that data (evidence) may be used to support the claim. The least sophisticated reasoning is considered to consist of a single claim or claims without any form of support. Given its in-depth and interconnected nature, the EBR-Discourse framework is not intended for use in real-time analysis of science classroom discourse; rather, it is intended for analyses of transcribed classroom discourse to assess the extent to which students are able to reason from evidence in a conversation in which teachers elicit their ability to do so. The framework is intended for use on portions of lessons where reasoning about science phenomena in classroom discussions is expected to happen. Within these discussion segments, related elements of reasoning are identified in what we call reasoning units, which are defined as coherent segments of reasoning that refer to the same claim, premise or both. Reasoning units can be of variable length, consisting of only a piece of a student or teacher speaking turn, or including a number of speakers across several minutes of classroom talk. Three possible sets of codes may be applied to each reasoning unit. The first set of codes combines elements of reasoning in each reasoning unit to determine the quality of reasoning the extent to which claims are backed up with data, evidence, and rules. The second set of codes focuses on a teacher s contribution in each reasoning unit; it identifies the extent to which teachers prompted students for elements of reasoning or provided these elements themselves. Finally, the third set of codes focus on the conceptual level of reasoning within each reasoning unit; these codes characterize the conceptual basis of claims being made. From this set of codes we capture the level of support provided by the teacher during the unit. The relationship between the unit of analysis and coding categories is shown in Figure 1. [Insert Figure 1 About Here]

REASONING IN SCIENCE CLASSROOM DISCOURSE 12 The following sections will focus first upon identifying the elements of reasoning, which are identified with the instrument first to determine the unit of analysis. Then, we will discuss the level of reasoning, teacher contribution to reasoning, and conceptual level codes. Finally, we will demonstrate how these codes can be applied in a sample transcript from a whole-class discussion. Elements of Reasoning The elements of reasoning (premise, claim, data, evidence, rule) as described in Paper 2 (this issue) describe basic functions of statements by teachers and students within classroom discourse about science phenomena. In classroom talk, the premise is often the subject of the sentence that contains the claim, while the claim is often the verb that describes what the subject has done, is doing, or will do. For example, the statement the boat will float contains a premise that is the subject of the sentence ( the boat ), as well as a claim that is also the verb in the sentence ( will float ). The backing provided for this claim-premise statement is often stated implicitly or explicitly as a because statement; that is, the statement the boat will float because of its shape contains a premise, claim, and backing ( because of its shape ). All statements of backing are classified as being data, evidence, or a rule. In this case, the backing is further classified as data, since it relies upon a characteristic or property of an object. Specific definitions for identifying the elements of reasoning are illustrated in Table 1. [Insert Table 1 About Here] To create units of analysis within a section of transcribed discussion, coders should begin by identifying all instances of claims, premises, and backing. Then, to identify each new reasoning unit, the coder should look for changes in the claim, premise, or both. To facilitate this process, the coder can refer to the claim-premise decision chart, shown in Table 2.

REASONING IN SCIENCE CLASSROOM DISCOURSE 13 [Insert Table 2 About Here] While these elements of reasoning are usually stated explicitly in writing, when analyzing classroom talk, we found many instances of implicit claims that were known to all participants in the discussion as a result of being a part of the ongoing social text (Aulls, 1998). Take, for example, the excerpt of transcript shown in Table 3, in which a premise is stated explicitly in the first line, and then referred to implicitly in the following two speaking turns by two different students. [Insert Table 3 About Here] In this excerpt, the first reasoning unit contains an explicit reference to a premise by the teacher, and the following statement by the student that contains a claim in reference to the implicit premise. The reasoning unit changes with the next student statement, which again refers to the implicit premise, but makes a new claim (the wood will sink). We found it helpful during the process of identifying claims, premises, and backing, to write a storyline that puts together the co-constructed statements from the teacher and student that are relevant to the ongoing process of reasoning in simpler form. The purpose of the storyline is to interpret the content of reasoning, not the structure, and serves the purpose of determining what the teacher and students have said as it relates to the reasoning prior to coding. More simply, the process of writing the storyline determines the official interpretation of the ongoing conversation. Once the reasoning units in the entire transcript have been identified, coders apply the two common sets of codes to the unit: Quality of reasoning and teacher support of reasoning. Quality of Reasoning Codes

REASONING IN SCIENCE CLASSROOM DISCOURSE 14 Based on our review of the literature, we established that much of the reasoning in science classroom discourse has been found to be of low quality. We therefore realized that we would need to develop intermediate levels along a continuum between high-quality reasoning and unsubstantiated claims, as proposed in Paper 2 (this issue). In so doing we also draw parallels between the levels of reasoning identified by Driver et al. (1994) and Carey et al. (1989). On this continuum, the least sophisticated discourse is conceptualized as consisting of single claim(s) without any backing (unsupported reasoning). This type of reasoning may also include circular or tautological statements as backing (e.g., the rock sinks because it sinks), or meaningless statements (e.g., the rock sinks because it wants to sink). Partial reasoning structures rely on data or evidence only. Those structures that reference only data or specific phenomena (phenomenological reasoning) as backing for a claim rely on single observations by students (e.g. the rock sinks because I saw it sink) or single properties (e.g. the rock sinks because it s heavy). More sophisticated reasoning is backed by evidence (relational reasoning) in the form of comparisons between properties (e.g. the rock sinks because its mass is greater than its displaced volume) or are summaries or datapoints (e.g., the rock will sink because in our test every rock sank). The most sophisticated reasoning is conceptualized by the full model of scientific reasoning shown in Figure 1, consisting of claims about phenomena that are supported by a generalizable statement about relationships between properties (or a rule); we call this rule-based reasoning. In addition to this statement of a rule, (empirical) backing such as reference to data collection may be used as backing.

REASONING IN SCIENCE CLASSROOM DISCOURSE 15 These four levels of reasoning are further explicated in Table 4; the far right-hand column provides a diagram illustrating how the elements of reasoning are related to each other at each level. [Insert Table 4 About Here] The quality of reasoning codes combine the components and processes elements of the model of scientific reasoning presented in Paper 2 (this issue). Table 4 also illustrates that in certain types of reasoning, explicit reference to all three types of backing are not necessary to be coded at a particular level. For example, we consider a claim and premise, backed by a rule, as an incidence of rule-based reasoning, rather than expecting students to go through the entire data to evidence to rule transformation in support of a premise and claim. Teacher Contribution to Reasoning Codes Since the discourse that occurs in science classrooms is to a certain degree supported and, in some cases, supplied by the teacher, we have created a second set of codes that indicate the extent to which the teacher scaffolded a particular reasoning unit. To that end, the framework includes codes for the teacher s contribution to each reasoning unit, in terms of which elements of reasoning the teacher prompted, and which elements the teacher provided. The teacher contribution to reasoning is coded according to the function of teacher statements in each reasoning unit; i.e. the extent to which teachers prompt for or provide elements of reasoning. Every component of reasoning (data, premise, data, evidence, or rule) that was either contributed (provided) or prompted (solicited) by the teacher is given a code within the reasoning unit; each code is applied only once. The different codes and examples are provided in Table 5.

REASONING IN SCIENCE CLASSROOM DISCOURSE 16 [Insert Table 5 About Here] Once an entire reasoning unit has been coded for teacher support, these codes combined with the quality of reasoning codes indicate the extent to which reasoning was provided or supported by the teacher. For example, a reasoning unit coded as featuring rule-based reasoning where the teacher provided a premise and claim and then prompted for a rule is quite different from a unit in which the students provided all elements. Conceptual Understanding as an Add-on Code The conceptual-level coding structure can readily incorporate modular add-ons addressing a range of additional variables relevant to the knowledge construction process. In our example below, we show a third possible code as an add-on, namely the conceptual level, which describes categories of domain-specific concepts that students and teachers incorporate into scientific arguments. Examples of other modules that have been used with this framework include conceptual explicitness, which describes the extent to which speakers say what they mean by concepts and their relations (Paper 6, this issue) and cross-argumentation, which captures the extent to which whole-class discourse amounts to an exchange of ideas between multiple students as opposed to mere student-teacher exchanges (Author, 2008b). We acknowledge the important linkage between students conceptual understanding, the quality of their reasoning and students conceptual understanding about phenomena. For example, Tytler & Peterson (2005) found that increases in students` ability to differentiate between theory and evidence were strongly associated with their conceptual understanding of the investigated phenomena. Von Aufschnaiter & Osborne (2008) illustrate this same covariance in cross-domain case studies of student reasoning. The emerging picture is that a more coherent understanding of the physical mechanisms underlying the investigated phenomena, such as

REASONING IN SCIENCE CLASSROOM DISCOURSE 17 evaporation, also supports students` use of more sophisticated reasoning, such as the comparison of patterns. In the case of sinking and floating, we created a set of categories, intended to become a common set of codes, that captured many ideas about density and sinking and floating from the literature (e.g. Piaget & Inhelder, 1942/1974; C. Smith, Carey, & Wiser, 1985; C. Smith, Snir, & Grosslight, 1992; E. L. Smith, Blakeslee, & Anderson, 1993) and our own research (Author, 2006b; Author, 2008c). These categories included the ideas that things sink or float because of shape, size, or weight; that water pushes up on objects to make them float, that there are sinking materials and floating materials, and that air makes things float. Unfortunately, we quickly found that differences in the curricula used in the different countries presented a constraint on the creation of a common conceptual coding system. The US curriculum was based upon a multilevel trajectory that started with alternative and naïve conceptions, then treated mass and volume independently, then together, and ended with explanations of sinking and floating based on relative density. In contrast, the German curriculum started with alternative conceptions but then intended students to arrive at an understanding of material kind; from there, the concepts of density, water pressure and buoyancy were treated. A major difference in the two curricula is that the German curriculum intended for implementation at the third grade incorporated buoyancy as its ultimate learning goal, whereas the US curriculum intended for middle school ended with relative density. Furthermore, conceptions considered to be scientifically accurate in the US curriculum were not viewed the same way by the German curriculum. For example, the US curriculum has the effect that volume and mass independently have on sinking and floating as a target conception whereas in the German curriculum, these conceptions are treated as unproductive if considered as a sole

REASONING IN SCIENCE CLASSROOM DISCOURSE 18 cause for floating and sinking. Similarly, the concept of hollowness was considered a prescientific conception in the German curriculum whereas it appeared as naïve conception in the US curriculum. The mismatch of conceptual codes is presented in Table 6. [Insert Table 6 About Here] In the end, we determined that the curriculum-dependent nature of these two sets of codes made it impossible to create a common coding system, and users of the framework each developed their own coding approach (see Papers 5 and 6, this issue). The Instrument in Action: Example of Coded Classroom Discourse Taken together, the sets of codes in the Reasoning in Science Classroom Discourse instrument provide information on the quality of reasoning in the classroom, the extent to which that reasoning is supported by the teacher or is performed by students independently, and the extent to which different claims and premises are begin negotiated in the course of the reasoning. Tables 7 and 8 present a sample of a coded transcript taken from a third-grade classroom in Germany to illustrate how the instrument is used. The tables split for the purpose of analysis a continuous transcript in which teacher/students first talk about whether a cube of wax will float in water, and then discuss whether a wooden board with holes in it will float. The discussion about the cube comprises a single reasoning unit (Table 7) while the wooden board discussion contains three different reasoning units reflecting students multiple claims about whether the board will sink or float (Table 8). [Insert Table 7 About Here]

REASONING IN SCIENCE CLASSROOM DISCOURSE 19 In the exchange presented in Table 7, the teacher presents students with two different object categories, the wax cubes and the wooden boards with holes, thereby providing students with the different premises about which they are to reason. She then prompts for the students to provide backing. In response, a student provides a claim based on a comparison of the mass of the object to water and, with additional support from the teacher, the students are able to provide evidence in support of that claim and with a more advanced level of conceptual sophistication. [Insert Table 8 About Here] In Table 8, the teacher provides a new premise (a piece of wood with holes in it) and asks for claims, and is met with two disagreeing students, neither of whom independently provides support for his or her claim. The teacher asks only one of those students to support her claim, and is provided with evidence in the form of a comparison of the mass of water and the board. Limitations and Challenges of the EBR-Discourse Framework Like any analytic tool, the EBR-Discourse framework is more appropriate for capturing some aspects of classroom discourse than others. It also creates its own set of practical challenges. In this section we discuss what we consider to be the most important of the limitations and challenges we have encountered so far. Reasoning From Evidence and the Process of Student Learning Increasing students capability to construct and defend scientific arguments is an instructional goal unto itself, but discussion and argumentation are also important avenues by which science concepts and principles can be learned (Brown & Campione, 1994; Driver Newton & Osborne, 2000; Lemke, 1990). In its bare form presented here, the EBR-Discourse Framework is not well suited to capture the potential contribution that discourse can make to the

REASONING IN SCIENCE CLASSROOM DISCOURSE 20 process of learning. Two limitations particularly stand out. First, the EBR-Discourse Framework does not address the social aspect of knowledge construction vis a vis the extent to which multiple participants consider and respond to one another s ideas. For example, Erduran, Osborne & Simon (2004) identified a series of levels of argumentative discourse that allows coders to capture the extent to which students are defending and rebutting different ideas. Second, it is does not capture the potential richness of student talk, including how explicit students make their thinking, or whether they generate their own expressions and examples of important ideas. While the framework can be modified to address both of these important aspects of discourse, there are tradeoffs in doing so, as we discuss below. Reasoning as disagreement: capturing the exchange of ideas in discussion. Table 8 shows a disagreement between two students about whether the piece of wood with holes in will sink or float. The first student, in Reasoning Unit 2, claims the board will float. The next student, in Reasoning Unit 3, claims it will sink. Prompted by the teacher in Reasoning Unit 4, Student 1 explains why it will float: it is that much lighter than water as a result of the holes. We could imagine a third student who might challenge Student 1 s claim that making something lighter necessarily makes it float higher. This type of disagreement, wherein students talk about multiple ideas in relation to one another, is crucial to the knowledge construction process within discussion and argumentation. While segmenting this one conversation into three different reasoning units allows us to examine the quality of student reasoning as associated with each separate claim, it simultaneously obscures the exchange of ideas between participants in the absence of some higher-level coding similar to Erduran, Osborne, & Simon s (2004) procedure. Given the reasoning unit as a grain size, the EBR-Discourse Framework cannot identify lively, intense

REASONING IN SCIENCE CLASSROOM DISCOURSE 21 discussion when it takes place; nor can it identify when discussion is little more than individual students responding to individual questions from the teacher in the classic IRE pattern (Mehan, 1979). It is possible to modify the framework s unit of analysis to allow researchers to more effectively grapple with of the exchange of ideas within discussion or argumentation. For instance, Author (2008b) used a unit of analysis of 1 minute portions of transcript on a large corpus of video data. Having done away with separate reasoning units for separate claims, these authors were able to add a module to the framework that identified reasoning units in which multiple students discussed the same idea, distinguishing these more vibrant discussions from those involving only 1 student and 1 teacher or those in which multiple students argued for unrelated ideas. However, because any single minute of transcript did not necessarily contain all related elements of reasoning (e.g. premise, claim, backing) their analysis of teacher contribution to student reasoning was necessarily less fine-grained than that presented here. Talk as an act of knowledge construction: evaluating the richness of student dialogue. Students learn by talking not just by parroting back what the teacher has told them to say but by expressing ideas in ways that make sense to them (NRC 1999). To give voice to a personally meaningful explanation is to associate new and prior knowledge and therefore to build conceptual coherence. Moreover, students explicitly and vividly describing science concepts can involve valuable metacognition. As they hear themselves talk, students can begin to identify gaps, conflicts, and other deficiencies in their understanding (Brown & Campione, 1994). We expect, in science inquiry, that conceptually rich discourse will involve a high level of argumentation. Yet the reverse is not necessarily true; that is, a discourse might encompass frequent appeals and references to evidence and yet be quite bereft of the active sense making

REASONING IN SCIENCE CLASSROOM DISCOURSE 22 that we value in the inquiry process. Thus the EBR-Discourse Framework cannot stand on its own as a measure of the overall quality of scientific reasoning. Science reasoning does not begin with bare facts but with theory-dependent observational statements. Teachers might present students with evidence, but they necessarily interpret it for themselves; unfortunately, the EBR video framework does not address such interpretation, which can be crucial to conceptual progress. Students interpretation of phenomena into their own language does not necessarily have a positive association with the quality of student reasoning from evidence. For example, in Table 8, reasoning unit 1 and reasoning unit 4 are both described as being evidence-based reasoning. Yet in the first reasoning unit, the student engages in a rich process of conceptualization, making the intensive aspect of lighter and heavier more explicit as he/she progresses through the discourse. Reasoning unit 4, by contrast, is not so explicit, and does not make the same progress. Both reasoning units show the same level of reasoning, but one is conceptually richer than the other. The potential for dissociation between students interpretation of phenomena and reasoning from evidence is the chief subject of Paper 6 (this issue). Difficulties with Definitions: Sorting Out Elements of Reasoning Another set of challenges we faced arose while applying the seemingly straightforward EBR framework to the messiness of classroom talk in two languages. One challenge was associated with the elements of the Toulmin argumentation scheme, and the second arose with the reasoning levels in the EBR-Discourse Framework itself. Claim or Premise? An established problem with Toulmin s scheme is that identifying what counts as a claim, premise, warrant, and backing can be difficult. In many cases, the premise and claim are indistinguishable, or what looks like a premise to one coder looks like a

REASONING IN SCIENCE CLASSROOM DISCOURSE 23 claim to another. Unfortunately, the structure of the EBR-Discourse Framework rests on being able to reliably identify these elements in order to establish the unit of analysis, as well as to be able to track the teacher s contribution to reasoning. To a certain extent, writing a storyline for each classroom discussion allowed us to establish an official interpretation of statements prior to coding them; however, the simple fact that these elements of logical argument could not, in all cases, be independently applied, calls into question the efficacy of looking at classroom discussion in this manner. When Is a Rule a Rule? A second challenge related to the EBR-Discourse Framework when applied to classroom videotape was determining whether or not rules presented by students in the absence of the data and evidence from which they were generalized were indeed rules, or were merely students remembering statements from class. When students mentioned rules without their supporting elements, we could either infer that students understood from whence that rule came, or treat the rule as a claim and therefore demote the level of reasoning from the highest level to the lowest level. The appropriate code could depend upon the instructional objective framing the discussion being analyzed. If the argument is supposed to justify the rule itself, then stating the rule without supporting data and evidence should be treated as low-level reasoning. However, if the rule was previously learned, and the lesson objective is to use the rule as a tool in an argument about a novel phenomenon, then it would be counter-productive to bring in the evidence for the rule. In the end, it is probably artificial to assume that students should return to the foundation of every rule they learn each time they construct a new argument, and as a result, the reasoning level applied within the EBR-Discourse Framework should be dependent upon the context and timing of the lesson being analyzed. Discussion

REASONING IN SCIENCE CLASSROOM DISCOURSE 24 As demonstrated above, the EBR Discourse framework provides us with information about the quality of reasoning provided for the claims advanced in a sample of classroom discussion intended to monitor students ability to reason from evidence. The framework allows teachers and researchers to determine the extent to which students are able to reason independently in whole-class conversations, and the extent to which they can only do so when supported by the teacher. The following two papers in this special issue (Paper 5, Paper 6) will each present an application of this common coding framework to a dataset in which students are engaged in a conversation centered around evidence for the purpose of the teacher monitoring and taking action on student thinking. Since each dataset was created independently before the collaboration that resulted in the creation of the EBR-Discourse Framework began, each paper takes a different approach to applying the framework to its unique dataset. Future research should explicitly build upon this framework so that data may be collected to more systematically apply the framework; for example, teachers might be made familiar with the EBR framework and then use it on-the-fly while guiding classroom discussions to actively raise the quality of student reasoning. This paper has presented the foundations of the instrument Evidence-based Reasoning in Science Classroom Discourse and has explained how it may be applied to transcripts of wholeclass discussions. However, as described above, the presentation of the instrument in this paper already raises issues for consideration in these analyses. Our highest level of reasoning included inductive and deductive reasoning, but we do not distinguish between the two; the framework could be revised to include epistemic categories similar to Jimenez-Alexandre et al. (2000). Future iterations of the framework may also involve an additional set of codes that can capture the extent to which actual argumentation is taking place between students (Osborne, Erduran, &

REASONING IN SCIENCE CLASSROOM DISCOURSE 25 Simon, 2004). In addition, going beyond simple frequencies based on the codes by mapping arguments based on the elements of reasoning and using a symbology similar to that employed by Resnick et al. (1993) could make other aspects of reasoning explicit that are obfuscated by the EBR-Discourse Framework. Future studies should continue to modify the framework as it is applied to other datasets to help us develop a further understanding for how reasoning can be supported in science classrooms.

REASONING IN SCIENCE CLASSROOM DISCOURSE 26 References American Association for the Advancement of Science. (1990). Science for All Americans. New York: Oxford University Press. American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy. New York: Oxford University Press. Aulls, M. W. (1998). Contributions of Classroom Discourse to What Content Students Learn During Curriculum Enactment. Journal of Educational Psychology, 90(1), 56-69. Author. (2006a). Furtak, Dissertation. Author. (2006b). Hardy et al. Author. (2008a). Shavelson, NSTA. Author. (2008b).Furtak et al., AME Author. (2008c). Yin et al, AME. Brophy, J. E., & Good, T. L. (1986). Teacher Behavior and Student Achievement. In M. C. Wittrock (Ed.), Handbook of Research on Teaching (3rd ed., pp. 570-602). New York: MacMillan. Brown, N. J. S., Wilson, M. R., Nagashima, S. O., Timms, M., Schneider, S. A., & Herman, J. L. (2008). A Model of Scientific Reasoning. Paper presented at the American Educational Research Association. Bruner, J. (1960). The Process of Education. Cambridge: Harvard University Press. Bruner, J. (1961). The Act of Discovery. Harvard Educational Review, 31(1), 21-32.

REASONING IN SCIENCE CLASSROOM DISCOURSE 27 Bybee, R. W. (2002). Scientific Literacy: Mythos oder Realität? In W. Gräber (Ed.), Scientific Literacy. Der Beitrag der Naturwissenschaften zur allgemeinen Bilduing. (pp. 21-43). Opladen: Leske und Budrich. Carey, S., Evans, R., Honda, M., Jay, E., & Unger, C. (1989). 'An Experiment Is When You Try It and See If It Works': A study of grade 7 students' understanding of the construction of scientific knowledge. International Journal of Science Education, 11, 514-529. Cazden, C. B. (2001). Classroom Discourse: The Language of Teaching and Learning. Portsmouth, NH: Heinemann. Driver, R., Asoko, H., Leach, J., Mortimer, E., & Scott, P. (1994). Constructing Scientific Knowlege in the Classroom. Educational Researcher, 23(7), 5-12. Driver, R., Newton, P., & Osborne, J. (2000). Establishing the Norms of Scientific Argumentation in Classrooms. Science Education, 84, 287-312. Duschl, R. A. (2003). Assessment of Inquiry. In J. M. Atkin & J. Coffey (Eds.), Everyday Assessment in the Science Classroom (pp. 41-59). Arlington, VA: NSTA Press. Duschl, R. A. (draft). The HS Lab Experience: Reconsidering the Role of Evidence, Explanation and the Language of Science. Duschl, R. A., & Gitomer, D. H. (1997). Strategies and Challenges to Changing the Focus of Assessment and Instruction in Science Classrooms. Educational Assessment, 4(1), 37-73. Edwards, D., & Mercer, N. (1987). Common Knowledge: The Development of Understanding in the Classroom. London: Routledge. Felton, M., & Kuhn, D. (2001). The Development of Argumentative Discourse Skill. Discourse Processes, 32(2&3), 135-153. Hempel, C. G. (1966). Philosophy of Natural Science. Englewood Cliffs, N.J.: Prentice-Hall.

REASONING IN SCIENCE CLASSROOM DISCOURSE 28 Jimenez-Alexandre, M. P., Rodriguez, A. B., & Duschl, R. A. (2000). "Doing the Lesson" or "Doing Science": Argument in High School Genetics. Science Education, 84, 757-792. Kawasaki, K., Herrenkohl, L. R., & Yeary, S. A. (2004). Theory Building and Modeling in a Sinking and Floating Unit: A case study of third and fourth grade students' developing epistemologies of science. International Journal of Science Education, 26(11), 1299-1324. Kelly, G. J., & Bazerman, C. (2003). How Students Argue Scientific Claims: A Rhetorical- Semantic Analysis. Applied Linguistics, 24(1), 28-55. Kuhn, D. (1993). Science as Argument: Implications for Teaching and Learning Scientific Thinking. Science Education, 77(3), 319-337. Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: University of Cambridge Press. Lemke, J. L. (1990). Talking Science: Language, Learning, and Values. Norwood, N.J.: Ablex Publishing Corporation. McNeill, K. L., Lizotte, D. J., Krajcik, J., & Marx, R. W. (2006). Supporting Students' Construction of Scientific Explanations by Fading Scaffolds in Instructional Materials. The Journal of the Learning Sciences, 15(2), 153-191. Mehan, H. (1979). Learning Lessons. Cambridge, Massachusetts: Harvard University Press. National Research Council. (1996). National Science Education Standards. Washington, D.C.: National Academy Press. Organisation for Economic Co-operation and Development. (2003). The PISA 2003 Assessment Framework: Mathematics, Reading, Science, and Promlem Solving Knowledge and Skills o. Document Number)