Classifying combinations: Do students distinguish between different categories of combination problems?

Classifying combinations: Do students distinguish between different categories of combination problems? Elise Lockwood Oregon State University Nicholas H. Wasserman Teachers College, Columbia University William McGuffey Teachers College, Columbia University In this paper we report on a survey study to determine whether or not students differentiated between two different categories of problems involving combinations problems in which combinations are used to count unordered sets of distinct objects (a natural, common way to use combinations), and problems in which combinations are used to count ordered sequences of two (or more) indistinguishable objects (a less obvious application of combinations). We hypothesized that novice students may recognize combinations as appropriate for the first category but not for the second category, and our results support this hypothesis. We briefly discuss the mathematics, share the results, and offer implications and directions for future research. Key words: Combinatorics, Discrete mathematics, Counting Introduction and Motivation Discrete mathematics, with its relevance to modern day applications, is an increasingly important part of students mathematical education, and national organizations have called for increased teaching of discrete mathematics topics in K-16 mathematics education (e.g., NCTM, 2000). Combinatorics, and the solving of counting problems, is one component of discrete mathematics that fosters deep mathematical thinking but that is the source of much difficulty for students at a variety of levels (e.g., Batanero, Navarro-Pelayo, & Godino, 1997; Eizenberg & Zaslavsky, 2004). The fact that counting problems can be easy to state but difficult to solve underscores the need need for more research about students thinking about combinatorics. One fundamental building block for understanding and solving combinatorial problems are combinations (i.e., C(n,k), also called binomial coefficients due to their role in the binomial theorem). Combinations are prominent in much of the counting and combinatorial activity with which students engage, and yet little has been explicitly studied with regard to student reasoning about combinations. This study contributes to our understanding of students reasoning about combinations, and, in particular, beginning students inclination to differentiate between typical combinatorics problems. This study addresses the following research question: Do early undergraduate students use binomial coefficients to express the solution to two different categories of combination problems? Theoretical Perspective Lockwood (2013; 2014) has argued for the importance of focusing on sets of outcomes in solving counting problems. In an initial model of students combinatorial thinking, Lockwood (2013) proposed that there are three inter-related components that students may draw upon as they solve counting problems: formulas/expressions, counting processes, and sets of outcomes. Formulas/expressions are terms involving numbers or variables that reflect the answer to a counting problem. Counting processes are the series of procedures, either mental or physical, in which one engages as they solve a counting problem. Sets of outcomes are the complete set of objects that are being counted in the problem, and the cardinality of the set of outcomes gives the

answer to the counting problem (Lockwood, 2013). Lockwood also pointed out that a given expression might reflect a counting process, and that various counting processes may impose different respective structures on the set of outcomes. For example, consider solving a problem like How many arrangements are there of the letters A, B, C, and D? A 4-stage counting process to solve this problem is to first place one of the four letters in the first position (there are 4 choices), then to place one of the three remaining letters in the second position (3 choices), then to place one of the two remaining letters in the third position (2 choices), and finally to place the last remaining letter in the last position (1 choice). Because at each of these stages the number of choices is the same and is independent of the result of the previous stages, we can multiply the number of options at each stage. 1 This counting process thus yields an expression of 4 3 2 1, which is 24. Note that this counting process also imposes a structure on the set of outcomes, namely organizing them according to first, then second, then third, then fourth letter. The lexicographic listing of the set of outcomes in Figure 1 demonstrates the set of outcomes that might be associated with the specific counting process. ABCD BACD CABD DABC ABDC BADC CADB DACB ACBD BCAD CBAD DBAC ACDB BCDA CBDA DBCA ADBC BDAC CDAB DCAB ADCB BDCA CDBA DCBA Figure 1 A lexicographic listing of arrangements of A, B, C, and D The point of this example is that a given formula or expression (4 3 2 1) may have a combinatorial process that underlies it (the 4-stage process described above), which in turn structures the set of outcomes in some specific way. Lockwood (2013) argued that it would behoove students to have flexibility in their counting, especially in understanding that different counting processes may organize the sets of outcomes in different ways, and vice versa. Lockwood (2014) emphasized the importance of sets of outcomes, and argues for a setoriented perspective toward counting. In this perspective, counting involves attending to sets of outcomes as an intrinsic component of solving counting problems (p. 31). In this paper, we frame our work from this perspective, namely that sets of outcomes are the key factor in determining what counting situation one is in, and thus what counting process and formula may be appropriate. That is, as we discuss combination problems, we consider a problem to be a combination problem if the set of outcomes can be appropriately modeled in a particular way (specifically, as sets of distinct objects), as opposed to being attuned to problem features like particular key words or contexts. The nature of our data is such that we are, at times, only able to examine a student s written expression and must make inferences about their counting process based on that expression. In this way, we draw on Lockwood s (2013) model and its position that counting processes can and do underlie formulas and expressions. Those underlying processes can suggest how students might be conceiving of the set of outcomes, if at all. 1 This is due to the multiplication principle, which is a fundamental yet subtle combinatorial idea. See Tucker (2002) for a statement of the principle and Lockwood, Reed, & Caughman (2015) for a more in-depth discussion of its mathematical subtleties.

Literature Combinations as a Unifying Combinatorial Topic Combinations, or binomial coefficients, are one of those unifying topics that connect a variety of combinatorial contexts and ideas. They are commonly used to count particular kinds of objects, namely k-element subsets of n-element sets, or, equivalently, the number of ways of selecting k elements from an n-element set. But this fact is applied meaningfully in a variety of important combinatorial settings, establishing combinations as a foundational big idea in combinatorics. Specifically, in addition to solving counting problems, these binomial coefficients also show up as entries in the rows of Pascal s Triangle as a model for solving multichoose problems, and as the coefficients in the binomial theorem, (x + y) n = C(n, k)x n y n k, because k=0 to find the coefficient of a given x k term we simply choose which k of the n binomials will contribute an x term. Because of this important relationship, combinations are known as binomial coefficients and arise in a number of mathematical domains. Finally, combinations appear in many combinatorial identities, and they necessitate combinatorial proof, which is another key topic in combinatorics. The variety of contexts in which combinations naturally arise points to the invaluable role that they play in the domain of combinatorics. Given how pervasive and fundamental they are, we were motivated to better understand how students might reason about and learn this foundational combinatorial idea. Combinations in Mathematics Education Literature There is much documented evidence for the fact that students struggle with solving counting problems correctly. Some reasons for such difficulty are that counting problems are difficult to verify (Eizenberg & Zaslavsky, 2004) and that it can be difficult to effectively encode outcomes in terms of objects one knows how to count (e.g., Lockwood, Swinyard, & Caughman, 2015b). We seek to address potential difficulties by focusing on better understanding students application and use of combinations which are fundamental in enumeration. Piaget & Inhelder (1957) studied students mental processes as they solved arrangement and selection problems, and they took a special interest in determining whether permutations, arrangements, or combinations would be most difficult for students. Dubois (1984), Fischbein and Gazit (1988), and Batanero, et al. (1997) have also investigated the effects of both implicit combinatorial models and particular combinatorial operations on students counting, again considering differences in reasoning about particular problem types such as permutations and combinations. We extend existing work that focuses on students mental processes of foundational combinatorial ideas, seeking specifically to explore the extent to which undergraduate students appear to distinguish between two categories of combination problems (we define these two categories in the following section). Other researchers have looked at students as they explore the variety of settings in which binomial coefficients arise. For example, Maher, Powell, and Uptegrove (2011) documented several episodes in which students of their longitudinal study make meaningful connections between binomial coefficients, certain counting problems, and Pascal s Triangle. More specifically, Speiser (2011) documented one 8 th grade student s reasoning about problems involving block towers, which can be solved as combination problems, and connections she made to the formula C(n,r). In a similar vein, Tarlow (2011) reported on eight 11 th grade students who could make sense of a well-known binomial identity using both pizza and towers contexts. These studies highlight the many connections that binomial coefficients afford in combinatorial settings, suggesting that it may be beneficial for students to have a sophisticated understanding n

of binomial coefficients that could facilitate these kinds of valuable connections. We build upon this work by Maher et al., (2011) by focusing on whether or not students see two distinct settings as both involving combinations. Our work also builds on a recent study by Lockwood, Swinyard, & Caughman (2015a) in which two undergraduate students reinvented basic counting formulas, including the formula for combinations. Based on the students work on combination problems, Lockwood, et al., (2015b) suggested the importance of being able to correctly encode outcomes combinatorially (by which they mean the act of articulating the nature of what is being counted by associating each outcome with a mathematical entity such as a set or a sequence). More specifically, the students could solve combination problems but not others, which was curious given their overwhelming success overall. In our current study, we were able to mathematically characterize a fundamental difference between some common combination problems based on characterizing a difference in their sets of outcomes. We wanted to explore further whether, for novice students, this distinction in the set of outcomes would pose a significant hurdle to their ability to recognize binomial coefficients as equally useful in the solutions to both situations. This resulted in us differentiating two different, different categories of combination problems and speculating about whether or not, for students, this difference matched their reality. We introduce and explore this distinction in the next section. Mathematical Discussion Classifying Two Categories of Combination Problems In this section we outline mathematical details of combinations, and we highlight a distinction between two categories of combination problems based on their sets of outcomes that is the focus of our investigation. A combination is a set of distinct objects (as opposed to a permutation, which is an arrangement of distinct objects). Combinations can also be described as the solution to counting problems that count distinguishable objects (i.e., without repetition), where order does not matter. The total number of combinations of size k from a set of n distinct objects is denoted C(n,k) and is verbalized as n choose k. 2 So, the binomial coefficient C(n,k) represents the set of all combinations of k objects from n distinct objects. As an example, combinations can be used to select, from eight (distinguishable) books, three books to take on a trip (order does not matter) the solution is C(8,3), or 56 possible combinations. In contrast, other combinatorial problems and solution methods, such as permutations, are frequently organized in relation to the different possible constraints see Table 1. Table 1: Selecting k objects from n distinct objects Distinguishable Objects (without repetition) Indistinguishable Objects (with repetition) Ordered Permutations n! = n (n 1) (n 2)... (n k +1) (n k)! Sequences n k = n! n # n "... n # $ k Unordered Combinations n k = n! k!(n k)! Multicombinations n k = k + n 1 n 1 In this paper we refer to combination problems as problems that can be solved using binomial coefficients, in the sense that parts of their outcomes can be appropriately encoded as sets of distinct objects. Sometimes this encoding is fairly straightforward, as the outcomes are very apparently sets of distinct objects. For instance, in the above problem of selecting three books from eight books to take on a trip, the books could be encoded as the numbers 1 through 8 2 The derivation of the formula for C(n,k) as n!/((n k)!k!) is not pertinent to the study; Tucker (2002) provides a thorough explanation.

(because they are different books), and the outcomes are fairly naturally modeled as 3-element sets taken from the set of 8 distinct books. Any such set is in direct correspondence with a desired outcome; there are C(8,3) of these sets. We call such problems problems. In other situations, fairly typical combination problems may still appropriately be solved using a binomial coefficient, but recognizing how to encode the outcomes as sets of distinct objects is less clear. For example, consider the Coin Flips problem (stated in Table 2). A natural way to model an outcome in this problem is as an ordered sequence of length 5 consisting of 3 (identical) Hs and 2 (identical) Ts, such as HHTHT. Solving the problem then becomes a matter of counting such sequences. One way to count the number of such sequences is to arrange all of the five letters (in 5! ways), and then divide out the repetitive outcomes based on the fact that there are three identical Hs (3!) and two identical Ts (2!) a solution of 5!/(3!2!). One could think of first treating all five of the letters as distinct and then un-labeling identical Hs and Ts. However, we note that the problem also can be (and frequently is) solved using a binomial coefficient but, in order to do this, the outcomes must be encoded appropriately as a set of distinct positions in which the Hs are placed. Given the five possible distinct positions (i.e., the set: {1, 2, 3, 4, 5}), the outcome HHTHT would be encoded as the set {1, 2, 4}. This sufficiently establishes a bijection between outcomes (sequences of Hs and Ts) and sets of the numbers 1 through 5 because every outcome has a unique placement for the Hs (the Ts must go in the remaining positions). In this way, the answer to the Coin Flips problem is simply the number of 3-element subsets from 5 distinct objects (i.e., positions 1 to 5), which is C(5,3). This gives an identical formula of 5!/(3!2!) and it is another way of solving the problem. 3 We call these problems, which can naturally be modeled as sequences of identical objects, but which can be encoded so as to be solved via a binomial coefficient, I problems (See Table 2). Table 2: Characterizing two different categories of fairly standard combination problems I Description Example problem Natural Model for Outcomes Basketball Problem. There are 12 athletes An unordered selection who try out for the basketball team {(1,2,3,4,5,6,7), of distinguishable which can take exactly 7 players. How (1,3,5,7,9,11,12), } objects many different basketball team rosters could there be? An ordered sequence of two (or more) indistinguishable objects Coin Flips Problem. Fred flipped a coin 5 times, recording the result (Head or Tail) each time. In how many different ways could Fred get a sequence of 5 flips with exactly 3 Heads? {(HHTTH), (HTHHT), (TTHHH), } In light of various ways of encoding outcomes that facilitates the use of combinations, we point out that it may seem that combinations are actually being used to solve two very different kinds of problems. The outcomes in the Books problem are clearly unordered sets of distinct objects, but the outcomes in the Coin Flips problem are actually ordered sequences (not unordered) of two kinds of indistinguishable (not distinct) objects (Hs and Ts). Combinations are 3 We can use Lockwood s (2013) language to describe this phenomenon, in which two different counting processes might yield the same expression. Indeed, the point is that just by looking at a particular formula or expression that a student writes for a problem, it may require some interpretation as to what their underlying counting process to solve the problem might have been. We can hypothesize and look for supportive evidence, but it is possible for multiple counting processes to yield identical expressions.

applicable in both situations, but we argue that there could be a difference for students in identifying both problems as counting combinations. Indeed, although both categories can be, and frequently are, thought of in terms of counting (choosing) sets, using combinations to solve I problems involves an additional step of properly encoding the outcomes with a corresponding set of distinct objects. We thus posit that problems may be more natural for novice students, more clearly representative of combination problems than problems. In spite of the widespread applicability of combinations, we posit that students may not recognize both categories of combination problems as problems involving combinations. This may be due in part to the fact that students tend not to reason carefully about outcomes (e.g., Lockwood, et al., 2015b), and because distinguishable and unordered are not always natural or clear descriptions of the situation or outcomes. However, we note that it is important and useful for students to be able to solve these I problems (and use combinations to do so) because combinations often arise as a stage in the counting process. Furthermore, using combinations in this way can facilitate productive and efficient solutions. For example, consider a problem such as Passwords consist of 8 upper-case letters. How many such passwords contain exactly 3 Es?. This problem can be solved by using combinations as a stage in the counting process first we select 3 of the 8 positions in which to place Es (there are C(8,3) ways to do this), and then we fill in the remaining position with any of the 25 non-e letters (there are 25 5 ways to do this). Thus, the twostage process yields an answer of C(8,3)*25 5. Here, we recognize that a I combination problem can help us complete the first stage of the counting problem. Importantly, realizing that a combination is useful in the first step makes the problem much easier than if one were to try to only to use permutations to do so. If we tried to answer the Passwords problem without first selecting places for the Es, and instead took a permutation approach, it would be possible, but it would require a large number of complex case breakdowns. 4 Thus, we argue that being able to solve I problems by using combinations demonstrates the utility of combinations as a powerful tool, and yet it also represents a sophisticated understanding of what combinations count, because it involves encoding the set of outcomes in a particular way. Given that the ability to solve I combination problems may allow students to solve a wide range of problems, and that it can reinforce a more complete understanding of what binomial coefficients can do, we are motivated us to investigate whether or not students actually respond differently to the two different problem categories. Methodology We designed two versions of a survey, and although the surveys contained a number of elements, we focus in particular on features of the survey that serve to answer the research question stated above. Each survey consisted of 11 combinatorics problems, and each problem was designed with categories in mind that included problem category (I or ) and complexity (, Multistep, or Dummy). 5 combination problems refer to those that can be solved using a single binomial coefficient, in the sense that their outcomes can be appropriately encoded as sets of distinct objects; multistep combination problems would require multiple binomial 4 For example, we would have to consider how many of the non-es are distinct and then arrange them. So we would have to account for cases in which a) all of the non-e letters are identical, b) exactly one of the non-e letters is identical (and the rest are distinct), c) exactly two of the non-e letters are identical (and the rest distinct), there are exactly two pairs of identical non-e letters, and so on. This process can generate the right answer, but it is complex. 5 We also coded the tasks according to other criteria that we do not report on in this paper, such as: sense of choosing (Active or Passive), and whether an object or process is to be counted (Structural or Operational).

coefficients in the solution (see Table 3 for problems in Survey 1 and the Appendix for Survey 2). The authors coded the problems independently before finalizing the coding for each problem. Each version of the survey contained the same number of problems of each category and complexity, as well as the same two Dummy problems to discourage students from assuming that every problem could be solved with a combination. Each problem was selected for one version of the survey with a companion problem in mind for the other version in order to compare responses with respect to the various coding categories. In this report, however, we primarily use the two surveys as additional support that the phenomenon observed is not limited to particular problems on one survey, but is consistent across a larger variety of these different categories of problems. Table 3: Survey 1 Description Dummy Dummy Multistep Multistep Problem There are 12 athletes who try out for the basketball team which can take exactly 7 players. How many different basketball team rosters could there be? There are 8 children, and there are 3 identical lollipops to give to the children. How many ways could we distribute the lollipops if no child can have more than one lollipop? There are 3 green cubes and 4 red cubes. Sam is making towers using all of the 7 blocks by stacking the cubes on top of each other. How many different towers could Sam make? Computers store data using binary notation - an ordered sequence of 0s and 1s. A particular piece of computer data is 95 digits in length, and it has exactly 12 1s. How many possible sequences fit this constraint? Stella is stacking ice cream scoops onto a cone. She has 3 scoops of chocolate, 5 of vanilla, 2 of pistachio, and 6 of strawberry. How many different ways can she stack all of the ice cream scoops onto the cone? In Montana, a license plate consists of a sequence of 3 letters (A-Z), followed by 3 numbers (0-9). How many different possible license plates are there in Montana? There are 12 points, all different colors, drawn on a sheet of paper (and no three points are on a line). How many different possible triangles can be made from these 12 points? Bob got a new job and is at a store looking for new ties. The tie rack has 196 different ties to choose from. In how many ways can Bob select 10 ties to buy? Fred flipped a coin 25 times, recording the result (Head or Tail) each time. In how many different ways could Fred get a sequence of 25 flips with exactly 11 Heads? There are 8 females and 10 males who would like to be on a committee. How many different committees of 6 people could there be if there need to be exactly 2 females on the committee? From an Olympic field of 15 athletes competing in the 100-meter race, how many different possible results could there be for gold, silver, and bronze medals? We targeted Calculus students because they were believed to have been likely to have encountered combinations at some point in their mathematical careers (perhaps in middle or high school) without having studied them in detail, making them informed but novice counters. Although we collected some additional demographic information about their previous mathematics courses and experiences with combinatorics, we have yet to incorporate this into our analysis. Overall, 281 people started the survey; however, many of these did not answer a single question on the survey, leaving n=126 people (65 for Survey 1, 61 for Survey 2), who responded to at least one of the combination questions. We included a reminder in the beginning

of the survey that offered a brief standard overview of combinations and permutations. The goal was for students to be reminded of the formulas and notation for combinations and permutations that they likely had encountered previously at some point in their education. The prompt for the combination problems asked participants to use notation that would suggest their approaches, rather than just numerical values. We gave this prompt because we wanted to be able to know what the student s counting process was, and so we would not have to make inferences based only on numerical responses. Specifically, the prompt was: Read each problem and input your solution in the text box. Please write a solution to the problem that indicates your approach. If you're not sure, input your best guess. NOTE: Appropriate notation includes: 9+20, C(5,2), 5C2, 21*9*3, 5*5*5*5=5^4, 8!, 8!/5! = 8*7*6 = P(8,3), C(10,2)*3, Sum(i,i,1,10), 12!/(5!*7!), etc. Only if you individually count all of the outcomes should you input a numerical answer, such as 35. In addition, for three of the problems, participants were prompted to expand particularly on how they understood the set of outcomes they were counting (see Appendix for this prompt.) The intent was to provide further evidence from the students about their counting process and the set of outcomes. Notably, however, many students had difficulties with these instructions, frequently providing numerical answers in their initial responses. We discuss further our analysis approach. In general, to investigate the research question we wanted to compare their solutions to the different categories of combination problems. Thus, we coded each response in three different ways. First, we coded whether the response was correct or incorrect. In this case, any problem the participants answered was determined to be correct or incorrect (with two exceptions: if the notation they used was not standard, e.g., C(3,5,2,6), which happened four times or their answer was unclear, e.g., a small fraction of 95 95, to where we could not judge correctness which happened five times). Importantly, many of the participants only included numerical answers, for which we could just confirm correctness, but not process. Therefore, secondly, for those participants that wrote a solution indicating their approach (i.e., followed the instructions) we coded the definite method that characterized their solution. If they used a combination, we coded whether they used the combination correctly (CC) or incorrectly (CI); if their answer involved a permutation, we coded it a P; if their answer was essentially multiplying numbers, we coded it M; if it only involved factorials that were not in a permutation or combination formula, we coded it F; if it involved exponents, we coded it E, if they just summed numbers, we coded it S; if they just used a single number from the problem, we coded it N; and if it was did not fall into any of the previous categories, we coded it O. Notably, for three of the problems, participants had an opportunity to explain more in relation to the sets of outcomes. For some participants who wrote numerical expressions, it was here that they explained their process. Therefore, when the numerical answer and their process matched up in these cases, we included a definite method based on their explanation. As mentioned previously, from some solution responses the method would not be completely clear. Particularly considering combinations, we had to determine the meaning of a response such as: 8!/(5!3!). Thankfully, there were not many such instances. For participants who appeared to be using the verbatim combination formula, such as 8!/(5!(8-5)!), we coded this as a combination; however, for participants who wrote 8!/(5!3!), if there was a clear indication from the set of outcomes responses or if they had used combination notation on other problems, e.g., C(20,10), we

presumed their response to be indicative of a permutation approach. 6 Thirdly, if numerical answers seemed to have a clear process such as the problem including the numbers 8 and 5, and participants answering 40 we coded the probable method that characterized their response. We consider this last coding involving probable codes as the best set of codes for our analysis for two reasons. First, some students gave correct numerical responses, such as 4,457,400, which we viewed as most likely indicative of using C(25,11), because coming up with this answer in some other way would be very difficult. This allowed us to include C(25,11) as their probable method to the extent possible, we wanted to give students the benefit of the doubt. Second, because so many responses were only numerical, this also allowed us to include some more responses in the analysis for which we could be fairly sure of their method. For the purposes of this paper, we limit our analysis only to the most basic combination questions the four simple problems and the three simple I problems on each survey. These seven problems provide us with the most basic comparison between their responses to these two categories of problems. So, using the third coding ( probable method), first we computed the proportion of instances on which participants used a combination approach at all for problems, and the proportion of instances on which participants used a combination approach at all for I problems, and compared these two proportions. We also computed the proportion of these responses that were correct uses of a combination approach. Notably, however, each individual participant answered up to seven such questions. Thus, for each individual, we secondly computed whether or not that individual had used a combination approach at all on at least one problem, and whether they had used a combination approach at all on at least one I problem. We also computed the proportion of participants that had used the combination approach correctly on at least one of each category of problem. This allowed us the ability to compare the proportion of distinct participants (not distinct problems) who had used a combination approach correctly on the differing problem categories. Thirdly, given that many participants methods were significantly off track i.e., many simply multiplied numbers in the problem without ever using a different method we limited our participants to only those that had used a combination approach on at least one problem. This allowed us to probe further whether participants that at least found a combination approach useful on one problem were differentiating between the two problem categories. For each proportion, we used a proportion t-test to determine whether the proportions were significantly different, and then used Cohen s h to determine the relative effect size of the differences. (We used standard cutoffs from Cohen (1988) of: 0.2, small effect; 0.5, medium; and 0.8, large.) Notably, when separating the analysis by survey, the results in every analysis were similar, and so we present the combined analysis across both surveys. Findings In this section we present the different analyses of our data, which all support the singular finding that students do indeed use a combination approach more regularly on problems than they do on I problems. We regard this result as indicating that from a 6 Indeed, one participant (1009), had correctly responded 20C4 to one problem, explaining it as, You choose any 4 out of 20. Order is irrelevant, and had responded 8!/((5!)*(3!)) on another problem, explaining it as, It's like having five dots and placing 3 dots in between the five. This is similar to arranging them in any way but not caring how they are arranged among themselves.

learners perspective there is a meaningful difference between these two categories of basic combination problems. Overall Participant Responses. Of the 126 participants, there were 117 of which we were able to give a probable approach code on at least one problem that they answered, yielding 380 total responses to problems for which we probably knew the participants approach. There were 116 participants for which we were able to give a probable approach code on at least one I problem, yielding 261 total responses to I problems for us to consider. Table 4 indicates the results. The data show that about 24% of the responses to problems used a combination approach, whereas only 16% of responses to I problems did a significant difference, but with a small effect size. Interestingly, most of the combination approaches to problems were correct (only 2/90 were incorrect), whereas about one-quarter of the combination approaches to I problems were incorrect (11/43 were incorrect). We see this as another indication that even if a student attempts to use a combination approach on I problems, they are doing so in ways that are, in fact, incorrect. Similarly, when isolating unique participants, we also find a statistically significant difference, with relatively small effect size, between the proportion of participants that were using a combination approach on compared to I problems. Table 4. Comparison of Overall Participant Responses I p-value Cohen s h On what proportion of responses is a combination approach used at all? On what proportion of responses is a combination approach used correctly? What proportion of participants used a combination approach at all on at least one problem? What proportion of participants used a combination approach correctly on at least one problem? 90/380 ~24% 40/261 ~15% p<0.01 88/380 ~23% 29/261 ~11% p<0.001 38/117 ~32% 24/116 ~21% p<0.05 37/117 ~32% 18/116 ~16% p<0.01 0.212 (Small) 0.324 (Small) 0.268 (Small) 0.385 (Small) Reduced Participant Responses. Since many participants never used a combination approach on any problem in the survey, as is evident from the overwhelmingly small proportions in Table 4, we reduced our participants to only those 42 who had used a combination approach on at least one problem either or I. These 42 participants yielded 153 responses to problems, and 114 responses to I problems. Table 2 indicates the results. Notably, for even these seemingly more knowledgeable participants there is a significant difference in the use of a combination approach to these two categories of problems. In fact, by reducing our population to only those who at least appear to have some idea that a combination might be a useful approach to solve a counting problem, we see larger effects in the differences. Given that these participants most closely match our desired population of participants students who had likely been introduced to combinations but not studied them

extensively, and who can use combinations appropriately in some settings we see these results as the most telling. Table 5. Comparison of Reduced Participant Responses I p-value Cohen s h On what proportion of responses is a combination approach used correctly? What proportion of participants used a combination approach correctly on at least one problem? 88/153 ~58% 29/114 ~25% p<0.001 37/42 ~88% 18/42 ~43% p<0.001 0.664 (Medium) 1.009 (Large) Discussion Despite the fact that both and I problems could be naturally encoded as combination problems, our findings suggest that the participants do not view the problems in this way. We found statistically significant differences in students use of combinations to solve versus I problems. In this way, our study offers quantitative evidence of what had been an anecdotally observed phenomenon. In terms of the model of students combinatorial thinking (Lockwood, 2013) and the setoriented perspective toward counting (Lockwood, 2014), this study suggests that students are not recognizing that outcomes of I problems can be appropriately encoded as sets of objects. That is, even though natural bijections exists that would allow students to leverage binomial coefficients in a variety of contexts, our research suggests that students are either not aware of this fact or are not able to use that bijection to encode outcomes effectively. Indeed, a large majority of students were simply not able to answer I questions in a correct manner, regardless of the method there were no other correct non-combination responses to the I questions, except one participant who correctly solved three I questions in a permutation manner. However, we want to acknowledge that it is not necessarily surprising that students would struggle to see this distinction. Indeed, familiar descriptions of unordered and distinct do not seem to apply at least in the most natural way to model the outcomes. Students can tend to associate counting with key words, specific contexts, and mantras like order doesn t matter, and they tend not to think about counting in terms of the outcomes they are trying to count (Lockwood, 2014). If this is a student s perspective on counting, it would follow that they would not be attuned to the importance of encoding outcomes and might not realize that they have the flexibility to encode outcomes in creative ways. Our study thus offers further evidence that students would benefit from focusing on the nature of the outcomes as the determining factor in what counting processes (and, ultimately, formulas) are most appropriate in a given situation. Conclusions, and Implications In sum, students may need additional exposure to combinations and may benefit from explicit instruction about how I problems can be encoded in a way that is consistent with problems. Generally, this point underscores a need for students to become more adept at combinatorial encoding (Lockwood, et al., 2015b). Encoding outcomes as sets is an inherent part of the field of combinatorics, but students may need particular help in making this

connection explicit. Also, these findings provide evidence for the fact that it may not be productive for students to be exposed to formulas initially if they are not pushed to understand those formulas. In terms of implications for instruction, then, we feel that teachers should explicitly direct students toward focusing on what they are trying to count. This means thinking of combination problems not exclusively as those problems whose outcomes can be encoded as sets of objects. Given how difficult (and seemingly unnatural) it is for students to encode outcomes of problems, instructors may need to give examples of ways to encode outcomes of I problems and to clearly establish relevant bijections. Discussing the relationship between how one models the set of outcomes and the pertinent solution approach may also be particularly meaningful in this context. For example, listing outcomes as 5 Hs and 3 Ts might lead to a permutation approach of 8!/(5!3!), whereas further encoding the outcomes in terms of the 5 distinct positions, for which three will be heads, might lead to C(5,3) as the natural solution approach. This is not to claim one approach as preferential over another, but we regard having both the flexibility to see different solution approaches as viable in this situation, as well as connecting sets of outcomes with particular solution methods, as highly important in developing an understanding of combinatorics. This might mean that instructors should first familiarize themselves with this distinction and to be able to understand and articulate what the distinction is between these two categories and why students might perceive them as different. Being able to use binomial coefficients flexibly and in a variety of settings can be a powerful tool for solving enumeration problems, but without a robust understanding including how and why I problems can be solved using them students may possess a tool they do not really understand how to use. As instructors, we should invest time and energy in helping students to understand this tool and the various ways in which it can be effectively used. There are natural next steps and avenues for further research. We plan to investigate more questions and hypotheses with the data we have, such as analyzing effects of demographic data and investigating other relationships and potentially contributing factors in students responses. We also want to explore the multistep problems more, and we wonder if perhaps the multistep problems might be similar in some way to the I problems, because both of these use a binomial coefficient as part of a process as opposed to the complete set of solutions. We could also see investigating similar kids of questions with counters with more experience than the novice calculus students, such as discrete mathematics or probability students. Our findings also indicate that further investigating students reasoning about encoding with combinations through in-depth interviews may give insight into the development of more robust understandings.

References Batanero, C., Navarro-Pelayo, V., & Godino, J. (1997). Effect of the implicit combinatorial model on combinatorial reasoning in secondary school pupils. Educational Studies in Mathematics, 32, 181-199. Dubois, J. G. (1984). Une systematique des configurations combinatoires simples. Educational Studies in Mathematics, 15(1), 37-57. Eizenberg, M. M., & Zaslavsky, O. (2004). Students verification strategies for combinatorial problems. Mathematical Thinking and Learning, 6(1), 15-36. Fischbein, E., & Gazit, A. (1988). The combinatorial solving capacity in children and adolescents. ZDM, 5, 193-198. Kavousian, S. (2008) Enquiries into undergraduate students understanding of combinatorial structures. Unpublished doctoral dissertation, Simon Fraser University Vancouver, BC. Lockwood, E. (2013). A model of students combinatorial thinking. Journal of Mathematical Behavior, 32, 251-265. Doi: 10.1016/j.jmathb.2013.02.008. Lockwood, E. (2014). A set-oriented perspective on solving counting problems. For the Learning of Mathematics, 34(2), 31-37. Lockwood, E., Reed, Z., & Caughman, J. S. (2015). Categorizing statements of the multiplication principle. In Bartel, T. G., Bieda, K. N., Putnam, R. T., Bradfield, K., & Dominguez, H. (Eds.), Proceedings of the 37 th Annual Meeting of the North American Chapter of the Psychology of Mathematics Education, East Lansing, MI: Michigan State University. Lockwood, E., Swinyard, C. A., & Caughman, J. S. (2015a). Patterns, sets of outcomes, and combinatorial justification: Two students reinvention of counting formulas. International Journal of Research in Undergraduate Mathematics Education, 1(1), 27-62. Doi: 10.1007/s40753-015-0001-2. Lockwood, E., Swinyard, C. A., & Caughman, J. S. (2015b). Modeling outcomes in combinatorial problem solving: The case of combinations. To appear in T. Fukawa- Connelly, N. Infante, K. Keene, and M. Zandieh (Eds.), Proceedings for the Eighteenth Special Interest Group of the MAA on Research on Undergraduate Mathematics Education. Pittsburgh, PA: West Virginia University. Maher, C. A., Powell, A. B., & Uptegrove, E. B. (Eds.). (2011). Combinatorics and Reasoning: Representing, Justifying, and Building Isomorphisms. New York: Springer. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: NCTM. Piaget, J., & Inhelder, B. (1975). The origin of the idea of chance in children. New York: W. W. Norton & Company, Inc. Speiser, R. (2011). Block towers: From concrete objects to conceptual imagination. In C. A. Maher, A. B. Powell, & E. B. Uptegrove (Eds.), Combinatorics and Reasoning: Representing, Justifying, and Building Isomorphisms (73-86). New York: Springer. Tarlow, L. D. (2011). Pizzas, towers, and binomials. In C. A. Maher, A. B. Powell, & E. B. Uptegrove (Eds.), Combinatorics and Reasoning: Representing, Justifying, and Building Isomorphisms (121-131). New York: Springer. Tucker, A. (2002). Applied Combinatorics (4th ed.). New York: John Wiley & Sons.

Appendix Prompt 2: On the previous page, you entered your solution to different combinatorics problems. On this page, we would like you to expand upon how your solution is related to the "set of outcomes" for a few of the problems. That is, we want you to list some (but not necessarily all) of the outcomes that you are counting. Explain your thinking for your solution and its relation to what is being counted. For example, for the problem, "If we have four distinct toy cars (Red (R), Blue (B), Green (G), and Yellow (Y)), how many different subsets of 2 of them are there?", your solution to the problem might have been: C(4,2). On this page, the intent is to expand on how that solution, C(4,2), relates to the set of outcomes. You might write something like, "The set of outcomes includes the following pairs of cars: BR, RG, GY, BG. I used the combination C(4,2) because the outcomes were "pairs" (2) from the 4 different colored toy cars. I did not include RB because this would be the same as BR in this case. Table: Survey 2 Description Problem You are packing for a trip. Of the 20 different books you consider packing, you are going to select 4 of them to take with you. How many different possible combinations of books could you pack? There are 9 justices on the Supreme Court. In theory, how many different ways could the nine justices come to a 7:2 vote in favor of the defendant? Stella is ordering an ice cream cone that is 8 scoops tall. She orders 5 chocolate scoops and the rest vanilla. How many different ways can the employee stack the ice cream scoops? There are 55 elementary students standing in a line. The teacher has 35 identical red balloons and 20 identical blue balloons, and gives each student either a red or a blue balloon. How many different outcomes are possible in this process? Dummy Dummy Multistep Multistep Sam is making towers from 3 green, 4 red, 2 yellow, and 8 orange blocks. Using all 17 blocks, how many different towers could Sam make? In Montana, a license plate consists of a sequence of 3 letters (A-Z), followed by 3 numbers (0-9). How many different possible license plates are there in Montana? There are 15 people in a room. Everyone shakes hands with everyone else. How many different handshakes take place? There are 250 kittens at a shelter. Sally is adopting 6 of them. In how many ways could she adopt 6 kittens? A professor writes a 40-question True/False test. If 17 of the questions are true and 23 are false, how many possible T/F answer keys are possible? There are 19 students in your class. How many ways are there to split the class into 3 different groups - one group of size 5, another of size 6, and another of size 8? From an Olympic field of 15 athletes competing in the 100-meter race, how many different possible results could there be for gold, silver, and bronze medals?