Activity Patterns of Pair Programming

Activity Patterns of Pair Programming Lan Cao Computer Information Systems Department Georgia State University Peng Xu Management Science and Information Systems Department University of Massachusetts Boston Abstract This paper reports results from an exploratory study investigating the activity patterns of pair programming using a student project conducted over a period of nine weeks. We found that pair programming engaged in activities that lead to more deeper-level thinking which may enable people to formulate a more complete mental model of the task. We also compared the activity patterns between different pair combinations. We found that there are differences in the patterns among different pair combinations. 1. Introduction Pair programming is a critical practice for Extreme Programming (Beck et al. 1999), one of the most popular agile methods. Pair programming is to allocate two developers collaboratively working on one task. Studies have shown that pair programming results in less defects in code and higher speed than solo programming (Williams et al. 2000c). In this paper we will investigate the activity patterns of pair programming using students as subjects. We focus on the interactions between the participants and compare the difference of the interactions among different pair combinations. This paper is structured as follows. In Section 2, prior empirical studies on the effectiveness of pair programming are reviewed. Studies on collaborative discovery are discussed. Section 3 describes the methodology for our empirical study. Section 4 presents the results from the study based on the analysis of the data, and finally section 5 presents the conclusions from the study. 2. Related Research 2.1 Effectiveness of Pair programming The need to empirically study the effectiveness of software development practices is well established. This is especially important in the context of agile software development as many of the early reports on the use of agile methods provide only anecdotal evidence on their applicability and effectiveness. Some of the basic elements of agile methods have been the focus of prior empirical research. For example, Nosek (1998) examined the role of collaboration in a study of 15 full-time, experienced programmers working on a challenging problem. This study concludes that collaboration improves both the programmers performance and their enjoyment of the problem solving process. Williams and Kessler (2000b) conducted a structured experiment comparing pair programming with solo programming in classroom settings. The results show that pair programming yields a better product, in less time and that developers using this approach enjoy the process more. Table 1 summarizes findings from these empirical studies. These studies share some common characteristics: first, the tasks used in the experiments are small in size. For example, the problem solving process in Nosek s study lasted only 45 minutes (Nosek 1998). Williams experiments (Williams and Kessler 2000a; 2000b; Williams and Upchurch 2001) used several small independent assignments during a semester session. Recent research suggests that task size and complexity have impact on the effectiveness of problem-solving practices (Barki et al. 1993; Chang et al. 1997). On one hand, for very small projects, the influence of development practices may not be significant enough to affect development process and outcomes. The benefits as well as shortcomings of the practices evaluated in such studies, such as pair programming, may not be very pronounced when the task is very small. On the other hand, concerns on the scalability of agile methodologies have also been reported (Boehm 2002). We have sought to balance these concerns by choosing a task that is complex enough for us to observe significant use of the practices implemented, but simple enough that they are clearly applicable for the use of agile practices evaluated. 1

Table 1. List of some studies on pair programming Article Method Subjects Measure Results (Nosek 1998) Controlled Experiment Professional programmers - Time - Number of test cases passed Pair programming results in better product and requires less time (Williams et al. 2000b) (Williams et al. 2000a; Williams et al. 2001) (Succi et al. 2002) Controlled Experiment Questionnaire Students - Time - Number of test cases passed Professional programmers Pair programming results in better product and requires less time. Developers enjoy the process more. - Job satisfaction Pair programming results in higher job satisfaction (McDowell et al. 2002) Controlled Experiment Students - Scores of assignment - Scores of exams Pair programming enhances student learning As described in detail in the next section, our study involved the development of an online application that lasted nine weeks. All the students were required to work with another student as a pair. Even though this task may not be considered very complex for experienced professional developers, in classroom settings, the task is considerably large and complex, especially in comparison to the tasks used in prior studies. 2.2 Collaborative Discovery The collaborative discovery method is part of a comprehensive methodology developed at Bell-Northern Research labs (Kenny 1989). Two subjects cooperate and work together to discover how to use a computer system. It has been found that collaborative discovery is beneficial in learning (Lim et al. 1997) and scientific discovery (Okada et al. 1997). Lim et al. (1997) find that codiscovery learning is more effective than self-discovery learning. Co-discovery subjects formulate mental models that provide greater inference potential than do those of self-discovery subjects. It is found that self-discovery subjects engage in more surface-level processes, while codiscovery subjects engage in more deeper level thinking that links lower level actions to higher level goals. Okada and Simon (1997) examined performance of pairs and single subjects in a scientific discovery task. They found that pairs performed better than the singles. Through the verbal protocols collected, they also found that pairs engage in more explanatory activities. Based on the findings, they argue that the observed performance advantage of pairs over selves is attributed to the nature of pair process, which forces more explanatory activities. In summary, literature shows that it is beneficial to adopt pair programming. However, limited empirical studies have been conducted to understand the process of this agile practice. Our study is designed to fill this gap. 3. Research Design We conducted a study using students in an undergraduate programming course. While working on a software development project, students used an agile approach which is code-centered and iterative. The subjects in this study worked on assignments and projects using several agile practices such pair programming, short iterations and evolving design. This paper reports only on the findings on pair programming. 3.1. Student subjects A total of 23 subjects participated in the study, including 4 graduate students, 14 seniors and 5 juniors. All were majoring in the field of computer information systems. Students in this class had taken at least one prior programming course. The average work experience for this group is five years, and the average experience in IT related fields is three years. 3.2. Project scope and process 2

Students were required to finish a course project, started in the middle of the semester. The project was to develop an online registration system for a university. The features of the system included major functionalities to support student registration. The students were asked to use the combination of HTML, Java and Java Servlet or Java Sever Pages. The project lasted nine weeks in total. There were four 2-week iterations. At the end of each iteration, the students were required to deliverable certain functionalities of the system. After the last iteration, all of the functionalities were developed and the students were given one week to work on other things such as documentation or improvement of the system. The project was not graded for the iterations, in instead, the grade is based on the final product that the students submitted. However, students performance in following the iteration plans is account for 10% of the total grade. 3.2. Data collected Beyond the programs that the students delivered, several other types of data were collected. 1) A questionnaire was used to collect student demographic and background data; 2) all students were asked to maintain a log of the effort they spent on each task; 3) after students tried pair programming in the assignments (before project started), feedback on this practice was collected; 4) a post-project questionnaire was used to collect students attitude towards the project and agile practices, and their confidence on their programs; and 5) interviews with selected students were conducted throughout the semester, in a format of conversation with students, emails, and student briefings after the video taping sessions. Even though all the students were asked to work in pairs, only six pairs continued to work in pairs. The other students started as a pair but for various reasons ended up working as traditional teams or individually. To investigate the pair programming practice, the six pairs were video taped when they worked on their projects. We examined the pair activities by protocol analysis. In this paper, we report findings from the analyzing the six pairs that represent three different pair combinations. The working sessions we taped of all pairs happened during the second iteration of the project. The reasons to choose the second iteration is as follows: 1 ) through developing the tasks required for the first iteration, the students had a better understanding of the problem domain and system requirements, and should focus mostly on coding; 2) this could give the students more time and practice to get used to coding in pair, as well as iterative development approach; and 3) this allowed the students to know each other better after working together in first iteration. 4. Results Pair programming was required to be used by all the students. Before the project started, students were required to practice pair programming on two small assignments. The purpose was to get the students used to pair programming and to help them to get to know their partners. The students were allowed to find their own partners to work with, and those who did not find partners after a period of time were assigned partners. Six groups formed on their own, five were assigned by the instructor, and one student worked alone. However, after the first iteration of the project, several problems were reported with implementing this practice. Some students reported schedule conflicts since working on such a complex project requires that the pairs meet multiple times each week, for several hours each time. Others reported communication problems. Some students who did not report problems to the instructor but they stopped working in pairs by themselves. Of the ten pairs that started with this practice, only six pairs worked together till the end. Thus, we had two groups of students, which we identify as the pair group and the non-pair group. Pair programming requires two developers work collaboratively on same task. Recent research (Williams et al. 2003) discusses the role of different pairing combinations such as expert-expert, expert-novice, expert-average, and novice-novice. There are unique problems and benefits relate to each pairing combination. In classroom settings, the pairs formed by students may be categorized as novice-novice. However, the students are different in their competence level in software development. The students who participated in the study in general are categorized to three groups based on their performance on assignments, exams, projects, and more importantly, the interaction with the instructor during the semester. High competent students are those who can quickly understand the contents of the course and relate it to her/his own tasks and solve a problem independently using previously learned knowledge. Medium competent students are those who can understand most of the contents of the course after studying the material or with the help from instructor. They can solve a problem by following a similar example. Low competent students can understand part of the contents of the course only and are not able to solve a problem by himself/herself. We expected that a student s competence level related to his or her background. We have collected each student s background data in class standing, courses previously taken, work experience in IT field, and working experience in general. After examining the students in each competence level and their backgrounds, we did not find strong correlations between the level of competence and prior experience. 3

As shown in table 2, at the end of the project, only six pairs worked out as pairs with 2 pairs in high-high combination, 2 pairs in medium-medium combination and 2 pairs in high-low combination. 3 pairs in high-low combination and 1 pair in medium-low combination did not worked out as pairs, as they ended up either working as a team or individually. Another 3 students worked alone most of the time during the project. Table 2 Number of Original Formed Pairs in Different Paring Combination Paring combination Pairs worked out High-High 2 0 Medium-Medium 2 0 High-low 2 3 Medium-Low 0 1 4.1 Coding Pairs not worked out The verbal protocols were transcribed and coded. In choosing a coding schema, we first considered the one used in (Lim et al. 1997) on computer learning. Lim proposed a model of human problem solving process in the context of human-computer interaction. The model categorizes the entire problem solving process into surface-level process (e.g., reading screen output) and deeper-level process (e.g., reasoning). As this model can be applied to general problem solving process, the specific nature of co-discovery in pair programming is not included in the model. Another model we examined was a model on collaborative discovery in scientific domain (Okada et al. 1997). This model captures the interaction of two participants in a specific problem solving process. Okada et al. (1997) developed a coding schema to examine the behavior of pairs and single subjects in a scientific discovery task. While this model is appropriate for analyzing explanatory activities in collaboration among pairs, it is specifically designed for tasks in a very unique problem solving process. It is difficult to apply the model directly to other domains. We start coding our protocol using the combination of the two schemas used in (Lim et al. 1997, Okada et al. 1997). Through analyzing the data, we developed a new schema for pair programming (Table 3). 4.2 Findings: Activity Patterns for Different Pair Combinations High-High There were two pairs in the class that were formed by students who were all very competent. All the participants Table 3 Coding Schema for Pair Programming Categories Leader s activities Ask for opinions Explanatory activities Critiques Summary of results Sub-activities Set up goal/tasks Summarize current status Formulating Strategy/Action Reconcile differences Ask for opinion Request for explanation Provide answer with explanation or justification Modify solutions Raise new problems Explanations Goal /task refinement Mental model extension reported that they liked pair programming very much, however, it is interesting to notice that in each pair there was one participant who reported that his/her partner was only moderate helpful, and the other participant reported that his/her partner was extremely helpful. Not surprisingly, both pairs passed most of the test cases and were very confident on their programs. All the participants also reported that working in pairs was more enjoyable than working alone. This combination is the best, as both the project and the developer benefit from pair programming. In the video tape of the two pairs, we found that the interaction pattern between the two participants is as follows: 1) One is the leader All participants were very competent. They had good understanding of the project domain, good programming and design skill, and good communication skills. They took turn typing the code. However, one participant took the lead of development all the time in the two pairs. At the beginning of each session, the leaders set up the goals and tasks, and frequently summarized their current status. They made decisions on what the next steps should be (Formulating Strategy/Action), refined the tasks, and reconciled their differences. In summary, the role of the leaders was to keep the work focused on the identified goals. In one pair, the more experienced participant served as the leader, and in the other pair, the leader was much more extraverted in personality traits than his partner. Table 4 summarizes the leaders activities with examples. The leaders might believe that they contribute more to the development than their partners even though both were capable of finishing the tasks. This explains why the leaders reported their partners were only moderately helpful. 4

Leader s Activity Set up goals for the session Set up tasks for next step Summarize current status Formulating strategy/action Reconcile differences Table 4 Leader s Activities Example I think we need to do is, the way we present: register, add, drop right now we got a single class and based on that, we can send to student file. So once you add you are going to add them in the database, and view and drop them, and add others. Now we re going to figure out how to pass these parameters. So actually we can combine these two so I log in to the faculty menu and student menu, and student menu will have these and you add these. 2) Ask for opinions It has been found that within a pair, the participants asked each other s opinions frequently. When a problem was raised, very often one participant asked for his/hers partner s opinions first, even though s/he might already have his own view on the issue. The focused discussion was initiated and the solution was identified from the discussion. The following example shows the pattern of asking a question ---focused discussion ---solution. A: what do you think? B: I think it would be easier to navigate around by having a menu. A: Oh, I think we definitely should so that, I was just saying should it be a separate servlet? B: It ll probably be easier because with whatever submit button they click on, it is always going to have multiple submit buttons A: Yeah, with add and drop button. B: so we d make this a separate servlet, or? A: I think we should B. Ok. One reason that they asked for other s opinions frequently is that the partners always provides valuable inputs to the conversation, as a result, they highly value each other s opinions. 3) Engage in explanatory activities According to (Okada et al. 1997), the better performance of pairs can be attributed to explanatory activities, which focus on the explanation and justification to ideas. It is found that the pairs engaged in explanatory activities very frequently. They asked for explanations on designs, ideas and codes. Request for explanations is different from asking for opinions as the former focuses on deeper level thinking (Lim et al. 1997) about a phenomenon, while the latter can be on both deeper or surface level thinking. The questions that initiate the explanatory activities are usually why, what if questions. During these interactions, learners actively engage in cognitive restructuring (Bearison 1982), which includes adding details to their existing cognitive structures, correcting their misunderstandings, filling in gaps in their knowledge, and/or reorganization their current knowledge structures (King 1989). Table 5 shows the explanation activities with examples. Cognitive Restructuring Adding details to existing cognitive structures Correcting misunderstandings Filling in knowledge gap/ reorganizing knowledge structures Table 5 Explanatory Activities Example A: Why is that? B: Add will pull all the class and all you need is to browse through whatever, check the add button, what we can do is just one screen, once you checked them all, you can go ahead to see what you have. A: Yeah. B: So once you add you re going to add them in the database, and you re going to view or drop them, and add others. A: Why do you think it is the same file? B: It is the same file name with the same size. A: Oh, you mean the database file. B: What if we not going that way? A: You don t have to mess around; you don t have to close the connection necessarily. B: But you have to figure out how to close the connection. A: I agree. I think it is not that easy as I thought we need to create a separate method It is noticed that in this combination, almost all the questions and requests were answered by the partners, and most of the time with explanations or justifications. According to (Okada et al. 1997), knowledge is generated from this interaction between the pairs. On the other hand, 5

the high quality answers encourage more conversation and keep the interaction healthy. 4) Critique partner s approach Many times there is an agreement between pairs. Some times a participant critiqued the partner s idea by disagreement of the approach or pointing out the problems in the design/code. The critique led to further explanation by the partner, modification of the solution, and raising new problems. Table 6 shows examples of each critique activity. 5) Summarize results The pairs in this combination summarized their current status frequently, which leads to goal or tasks refinement, and mental model extension. Table 7 provides examples of summary of results and the activities it leads to. Activities Critique Modify solutions Raise new problems Explanation Activities Summarize results Goal or tasks refinement Mental model extension Table 6 Critique Activities Examples B: In that way, instead of having to write all the connection stuff each time, we could have a method to do it. A: The problem is that we close it right here, it won t let you not close it. B: What if we have another method called Close_connection? A: Or we could just have a method that opens the two. B: But how can the method be accessed? A: It is available to all the classes as long as it is public. Table 7 Summarize Results Examples A: (read the sheet) we have developed a registration page, displaying all the available courses B: so this is where we are going to send that, in place of where we are sitting now, welcome student registration, we gonna send it to another servlet? A: Ah, what do we want to do? I have a picture here. Let s think about this. All right. Now we are accepting the log in from the Medium-Medium There were two medium-medium combinations worked as pairs. One pair reported that they really liked pair programming and the partners were very helpful. They passed 78% of the test cases. They thought working by pairs was more enjoyable and they were very confident of their work. The other pair provided conflict feedback. One participant reported that he liked pair programming and his pair was very helpful, while the other participant did not like it and commented that his partner did not help. As a result, the project was incomplete and they passed only 56% of the test cases. Analyzing the videos of pairs in this category, we found there are similarities as well as differences in their activity patterns compared to the High-High combination. 1) One is leader Both participants of the first pair were hard working. They had good understanding of the problem domain and good communication skills. Both of them relied on concrete examples to guide their development. Once those examples were not present, they frequently encountered problems. The second pair in this combination composes a participant who worked very hard, and a participant who invested less effort than his partner. As in the high-high combination, one participant took the role of leader in each pair. In the first pair, both of the participants were novice in software development. However, there were obvious differences in their personalities. The leader can be described as an extravert while her partner is more like an introvert. For the second pair, the hardworking participant was the leader. The leaders involved activities that were different from those in the high-high pair. For example, instead of trying to focus on the goals, the leaders in the mediummedium pairs focused more on details and seldom adjusted the tasks. Another difference is that the leaders directed the action of their partners frequently, sometimes with no justification. For example, one leader asked her partners to take some actions on the code: Just get rid of all section. Just try it. Let s try from the beginning from the homepage. Add this phase back. One reason for that might be that many times the leaders took arbitrary actions without fully understanding the problem. Another reason might be that the leader didn t expect his/her partner to help with better solutions when they encountered a problem. 2) Ask for opinion Similar to that in the high-high combination, the participants in the medium-medium combination asked for each other s opinions frequently. However, many times the questions were in surface level. As a result, focused discussions could not be initiated as it is in highhigh pairs. The following example shows the pattern of asking a question ---simple answer solution. 6

B: does this have to be exactly as it is in the database? A: it is. B: you sure it is? A: yeah. (look at the note) this is the one that the teacher gave, and I checked it. We can see that knowledge was not generated from this interaction. However, the process still helped clarify some ambiguities, and reach solutions. For that reason, they valued each other s opinions as they could help each other on solving some of the problems. 3) Engage in explanatory activities Compared to the high-high pairs, many times the medium-medium pairs failed to provide answers to the request for explanation. The participant replied frequently as I don t know, It makes no sense. 4) Summarize the results The pairs seldom summarized the results. Instead, they focused on the details of the problems they encounter. High-low Two pairs of this combination worked successfully and three pairs ended up failing on pair programming. However, all the five participants with high level of competence reported that they didn t like this approach and their pairs were not helpful at all. None of them reported that pair programming was enjoyable. However, they were still very confident of the programs they developed. In contrast, all the five students with low levels of competence in this high-low combination reported that they liked pair programming, their partners were very helpful. They also reported that they enjoyed the process and are also confident of the resulting programs. All the five pairs performed very well, the two successful pairs passed 78% and 89%, respectively, (average 83%) of test cases and interestingly enough, the two non-pair groups performed slightly better by passing 89% (89%, 78%, 100% respectively) of the test cases. So there is a clear pattern in the high-low combination: the highly competent participants do not gain much in pair programming and they don t enjoy working with their partners who are much less competent. The participants with low competence like this approach since they have a partner that they can rely on and accomplish tasks that they can not complete by themselves. It is interesting to find that the three teams that gave up pair programming performed better than the two teams that continued with pair programming. We have taped one working session of two pairs in this category. 1) One is leader One participant was very competent with good understanding of the project domain, good programming and design skills, and good communication skills. The other participant had problems understanding the problem domain as well as the logic of the program. During a onehour working session, the competent participants took the lead of development all the time in the two pairs. The leaders explained to their partners the goals and tasks at the beginning of the session. The leaders made decisions on what the next steps should be (Formulating Strategy/Action). The leaders also explained almost every action and the code in great detail to their partners. 2) Ask for opinions frequently At the beginning, the high competent participants asked their partners opinions frequently. However, most of the time, they did not obtain any valuable information. The low competent participants either repeated the questions, or say something that is not relevant to the question. For example, A: I think the quota has to be inside the semicolon, right? B: yes. A: But what happen when it goes to multiple lines like this? Doesn t it matter? B: SQL basically, that one should end with... you know Soon the high competent participants stopped asking for their partners opinion. Sometimes when the low competent participants provided opinions on certain issues, the high competent participant either agreed without serious consideration or ignored their partners and continued their own thoughts. For example, A: I think I can just do it on B: You changed the register A: So this prints the B: table, I mean the headers of the table. A: OK, and now we have to loop through the records sets, right? On the other side, the low competent participants asked questions for understanding and clarification frequently. They tried to learn from their partner through this way. 3) Engage in explanatory activities To make sure the low competent partners understand the program, the high-competent participants explained most of the actions they took. Interesting enough, the explanations also led to the benefits of explanatory activities even though the explanations were not a response to a request from the partners. Through 7

explaining the design and code to their partners, the high competent participants kept adjusting the goals and refining the tasks, as well as reorganizing their knowledge structures. Table 8 shows the cognitive restructuring involved explanatory activities. However, from the data we can see that the high competent participants had to spend extra time and effort trying to educate their partners, which in turn, distracted them from focusing on the development. That explains why teams that gave up pair programming performed a little better than teams that continued with pair programming. 4) Critique partner s approach There was not much critique going on in this combination. The low competent participants always agreed with their partners, and the high competent participants tended to ignore the comments of their partners. Table 8 Explanatory Activities Cognitive Restructuring Adding details to existing cognitive structures Correcting misunderstandings Filling in knowledge gap reorganization knowledge structures Example A: Oh, because that selects columns what we bring out, I didn t think about that, it should show the max capacity I think what it gonna do is to bring out the max capacity, it is gonna show that as a column and count of registration ID does it, it is pulling from three tables, from this table, the section table. B: Computer number then what, student name? A: Right now it is not about students, it is just the courses B: Where is this one? Take the quote out. A: Take the quote out? I don t know. A: No we need the quote. See, hers has it. When it is in ASP, it has to have quote. Because in SQL or Access, when it created a query, it put quotes around it. 5) Summarize the results The pair also did not summarize their status. 4.3 Summary of Patterns of Activities for Pair Combination Our study shows that a participant likes to pair with someone better than or at least as good as him/herself. In organizational settings, where pairs are rotated frequently, the effects of pair combinations on project and on developers are likely include all entire range discussed above. However, the decision to use pair programming even with less desirable combinations described above may be made for other considerations such as facilitating organizational learning and knowledge sharing. For example, even though the projects might suffer from high-low combination, the training and knowledge sharing between the partners can benefit the project team and the organization in long run. Table 9 summarizes the activity patterns for different pair combinations. Table 9 Summary of Activity Patterns for Different Pair Combinations Activities High-High Medium- Medium One leader Leader Leader focus focuses on on the keep the details; development direct process on partner s the targeted action with tasks no justification Ask for opinion Explanatory activities Frequently, and this leads to focused discussion. Frequently Frequently but no focused discussion followed. More surface-level interaction. Frequently, but many times the partner fail to provide explanations or answers to the requester. Critique Frequently Frequently No Summary Frequently Seldom No of results 5. Conclusions High-Low The high competent participant is the leader who explains the code in detail to partner. Only at the beginning Frequently, through explaining the details to partner 8

In this study, the activity patterns of pair programming were investigated. We found that pair programming engaged in activities such as asking for opinions, requesting for explanations, critiquing partners approaches and summarizing current status. These activities lead to more deeper-level thinking which may enable people to improve/revise a mental model they are currently holding and formulate a more complete mental model of the task (Lim et al. 1997). These patterns can help explain the benefit of pair programming. We also compared the activity patterns between different pair combinations. In this research, three combinations were investigated, i.e., high competent high competent, medium competent-medium competent and high competent-low competent. We found that there are differences in the patterns among different pair combinations. The high-high combination involves more deeperthinking activities than the medium-medium level combinations, while the high-low combination has the least interaction between participants. Generally speaking, the high-high pairs enjoyed pair programming experience more than the other pairs. Deep knowledge was generated when the participants in the high-high pairs interacted with each other. The effectiveness of pair programming was achieved. The performance of the medium-medium pairs was not satisfying. Very limited knowledge was generated. The performance of the high-low pairs was not satisfying. The high competent participants did not enjoy the practice while the low competent participants benefited from the practice. Even though the interaction between the participants in this combination is minimum, it is interesting to find that by trying to explain her actions and code to her partner, the high competent participant engaged in explanatory activities frequently. As a result, the pairs benefited from these activities. Another finding is that there is a leader in each pair. We found that who is the leader depends on the factors such as competence level, experience and personalities. The study suggests that different combinations in pair programming achieve different outcomes. Companies that adopt pair programming should carefully design their pair combinations according to their desired goals. For instance, if the goal is to transfer knowledge and train new personnel, high-low pattern could be used if the resource allows. Also the success of the high-low pairing depends on factors such as the personality of participants, schedule constraints, and quality expectations. For example, under a situation where collaboration in the team is not widely accepted, the schedule is tight, and the quality expectation is high, high-low pattern might be avoided. High-high pairing works best both for knowledge generation and quality improvement. Mediummedium pairing does not show interesting benefits in terms of both knowledge generation and quality improvement, practitioners should be cautious about this pairing strategy. As our study used students as subjects and was based on a small sample, the results can not be readily generalized to the use of pair programming in industrial settings. However, this study is exploratory in nature, and provides some interesting findings that need to be further evaluated in detail in future research. References Barki, H., Rivard, S., and Talbot, J. "Toward an Assessment of software Development Risk," Journal of Management Information Systems (10:2) 1993, pp 203-225. Beck, K., Hannula, J., Hendrickson, C., Wells, D., and Mee, R. "Embracing change with extreme programming," IEEE Computer (32:10), Oct 1999, pp 70-77. Bearison, D. J., "New Directions in Studies of Social Interactions and Cognitive Growth," in F. C. Serafica (Ed.), Social-Cognitive Development in Context, Guilford, NY. 1982 Boehm, B. "Get Ready for Agile Methods, with Care," IEEE Computer (35:1) 2002, pp 64-69. Chang, C.J., Ho, J.L.Y., and Liao, W.M. "The Effects of Justification, Task Complexity and Experience/Training on Problem-Solving Performance," Behavioral Research in Accounting (9) 1997. King, A., "Verbal Interaction and Problem-Solving Within Computer-Assisted Cooperative Learning Groups," Journal of Educational Computing Research (5:1) 1989, p1-15. Lim, K.H., Ward, L.M., and Benbasat, I. "An Empirical Study of Computer System Learning: Comparison of Co-Discovery and Self- Discovery Methods," Information Systems Research (8:3) 1997. McDowell, C., Werner, L., Bullock, H., and Fernald, J. "The Effects of Pair-Programming on Performance in an Introductory Programming Course," Proceedings of SIGCSE 2002., 2002. Nosek, J.T. "The Case for Collaborative Programming," Comm. ACM (41:3) 1998, p 105 108. Okada, T., and Simon, H.A. "Collaborative Discovery in a Scientific Domain," Cognitive Science (21:2) 1997. Succi, G., Marchesi, M., Pedrycz, W., and Williams, L. "Preliminary Analysis of the Effects of Pair Programming on Job Satisfaction," Fourth International Conference on extreme Programming and Agile Processes in Software Engineering (XP2002), 2002. 9

Williams, L., and Kessler, R. Pair Programming Illuminated Addison-Wesley, 2003. Williams, L., and Kessler, R.R. "The Effects of 'Pair- Pressure' and 'Pair-Learning' on Software Engineering Education," Conference of Software Engineering Education and Training, 2000a. Williams, L., Kessler, R.R., Cunningham, W., and Jeffries, R. "Strengthening the Case for Pair- Programming," IEEE Software (July/Aug) 2000b. Williams, L., and Upchurch, R.L. "In Support of Student Pair-Programming," 2001 SIGCSE Conference on Computer Science Education, Charlotte, NC, 2001. 10