A Pair Programming Trial in the CS1 Lab

A Pair Programming Trial in the CS1 Lab Shamim Khan 1, Lydia Ray 2, Aurelia Smith 3, Angkul Kongmunvattana 4 TSYS School of Computer Science Columbus State University Columbus, GA 31907, USA { 1 khan_shamim, 2 ray_lydia, 3 smith_aurelia, 4 kongmunvattana_angkul}@colstate.edu Abstract The benefits of pair programming have been recognized for some time now. Numerous attempts have been made to introduce pair programming in the classroom to improve student learning and performance. A trial introduction of this method in the introductory Computer Science (CS1) lab-based class is reported. The noticeable improvement found in student performance is in agreement with most previous studies. This study also explores student perception of this collaborative approach to learning, and some administrative issues. Keywords pair programming; collaborative learning; pair formation; driver-navigator model; student perception I. INTRODUCTION Motivation generated in the freshman year plays an important role in undergraduate student retention and progression. According to one study, 60% of students who abandon their course of study do so in their first year [1] at the university. Student failure and withdrawal rates in the introductory Computer Science (CS1) course are unacceptably high in most institutions more than 30% in the US according to a nationwide survey [2]. At Columbus State University (CSU), the average student success rate in CS1 over the last five years (2004 2009) has been around 53%. Student success in CS1 is decided largely by their success in learning problem solving using computer programming. Student performance in classroom exercises and assignments shows a large number of students struggling when it comes to understanding core computing concepts and essential features of programming languages. Lack of success leads to students loss of confidence and many simply give up. In recent years, a number of approaches has been tried to address the issues of both low CS1 enrollment and success rates. These include the use of more attractive interactive programming languages and IDEs (e.g., Alice, Greenfoot and Python), media computation [3], robotics [4], computer games [5] as well as pair programming. This paper reports a trial of the pair programming approach in the CS1 lab class over several semesters at CSU s School of Computer Science. II. PAIR PROGRAMMING IN INDUSTRY AND THE CLASSROOM The origin of pair programming is in extreme Programming (XP) - an emerging software development methodology intended to improve productivity. In software industry pair programming, production code is written collaboratively with a partner, where two programmers sit side by side and share a computer. Anecdotal evidence suggests that two programmers can be more than twice as fast and can think of more than twice as many solutions than the two working on their own. Also, constant code reviews, an inherent part of pair programming, results in higher defect prevention and removal leading to a higher quality product. As reported in a survey of professional pair programmers by [6], all participants agreed that they had more confidence in their solution when doing pair programming than when they worked alone. About 96% of them agreed that they enjoyed their job more when programming in pairs. In an experiment at the University of Utah [6], students in two courses - Collaborative Development of Active Server Pages (20 juniors and seniors) and Software Engineering ( 41 juniors and seniors) were divided into two groups paired and non-paired. Reportedly, students in pairs passed 15% more of

instructor s test cases compared with those working individually. Ever since the establishment of pair programming as a useful technique in professional software development, educators have been trying to understand its potential benefits in introductory computing courses. In 2002, a study was conducted on the effectiveness of pair programming on freshman learning at the University of Santa Cruz [7] showing improvements in the quality of completed programming assignments as well as the time spent in completing these assignments. A number of studies thereafter have also shown the positive effect of pair programming on student learning and the quality of code produced by students [8,9,10,11]. However, at least one study [12] demonstrated some negative aspects of pair programming. According to this study, students who used pair programming did not write significantly better programs than students who worked alone. Most students were of the opinion that the key to success in pair programming was to get a good partner. III. PAIR PROGRAMMING AT CSU In an effort to improve student retention and progression rates in Computer Science, as well as to attract non-computer Science majors to this discipline, it was decided to investigate pair programming as a teaching methodology. As part of a trial involving the CS1 lab class, students were paired together during the first week of the semester. A. Formation of Pairs A student s motivation to work collaboratively depends to a large extent on compatibility with his/her partner. Accordingly, a good deal of attention has been focused on the strategies for forming pairs. The following pair-formation methods have been suggested in the National Center for Women & Information Technology (NCWIT) pair programming-in-a-box resources [13]: - Pairs based on levels of programming experience - Pairs formed randomly - Pairs formed by students with instructor guidance The students involved in this study were all freshmen, with most of them in their first semester at the university. The traditional approach of asking students to find their partners to form a pair was not adopted since the students hardly knew each other, especially during the first week of the semester. This decision was later borne out by our survey results (see below) which indicated possible difficulty faced by students in this approach. Instead of forming pairs randomly, we decided to form pairs based on individual programming experiences. To assist the pairing of students, they were asked to do a selfevaluation of their programming skill in a survey carried out on the first day of the semester. An attempt was made in this process to pair students with comparable programming experience and skills by avoiding strong-weak and weak-weak pairs as far as possible. This was based on the idea that putting two weak students together might put these students at a relative disadvantage compared with pairs with a different mix. Strong-weak pairing also was avoided with the expectation that such a pairing would result in the more skilled student dominating over his/her weaker partner, thus making it difficult to maintain the spirit of learning through equal participation. B. The Pair programming Model Students in each pair worked collaboratively on each assignment. They were asked to assume driver and navigator roles and switch them regularly. The driver controlled the mouse and keyboard to enter the code, compile and run it. The navigator actively examined the driver s work for syntax and logic errors, and any deficiencies in the design, or for possible designimplementation mismatch. He/she also looked up resources and thought of alternatives. The driver and navigator would switch roles every 20 minutes or so, ensuring they worked as a team and obtained the benefit of acting in each role. The pair would continue working in this manner until the programming assignment was complete. The issue of assessing work done by students in a pair is a difficult one. The simplest approach is to award both students the same grade, although ideally, in a group effort, individuals should receive grades based on the amount of effort they put in. Assessing individual contribution, for example, through interviews, can be time consuming and difficult to

manage. We adopted an approach built partially on peer review. Students in a pair were required to submit their work jointly but respond to a few peer review questions independently. Both students in a pair received the grade for their joint effort after it had been weighted by mutual peer evaluation. The evaluation questions used are given in Table 1 below. Students had to submit answers to these questions with each assignment work using a 5-point Likert scale, with 5 corresponding to strongly agree, 3 to neutral, and 1 to strongly disagree. Each student s grade was scaled by the average of his/her partner s responses to these three questions. TABLE I. PEER EVALUATION QUESTIONS 1. My partner did his/her fair share of the work 2. My partner was cooperative 3. My partner was willing to switch drivernavigator roles IV. RESEARCH OUTCOMES The two broad objectives of our investigation were: (1) to find out if pair programming is beneficial to students and (2) to assess student perception of pair programming. To ascertain the impact of pair programming on student performance, course grades earned by students in three CS1 classes taught with pair programming over two semesters during 2009-2010 were compared with the grades from classes offered without this approach over the previous five semesters during 2006-2008 (the CS1course did not have a separate lab-based component before 2006). It was assumed that improved grades would indicate an overall improvement in students understanding of the material covered in the class as well as their programming skills. Table 2 below shows the percentage of students failing to obtain a satisfactory grade. The DWF value represents the fraction of the total number of students who were unable to obtain an A, B or C grade, and it includes students who withdrew from the course late in the semester, presumably after failing to keep up with the class. The last three rows of this table correspond to data from the three groups of students taught with pair programming. Figures in table 2 shows that, based on available data, student performance in classes with pair programming is significantly better, with DWF rates in the range 7%- 45%, compared with performance in classes where students worked on their own, with these rates in the range 20%-57%. TABLE II. STUDENT DWF GRADE PERCENTAGES IN CLASSES WITH AND WITHOUT PAIR PROGRAMMING Semester DWF (%) Fall, 2006 57% Spring, 2007 20% Fall, 2007 30% Spring, 2008 47% Fall, 2008 27% Spring, 2009 a 45% Fall, 2009 (section 1) a 7% Fall, 2009 (section 2) a 21% a. Classes offered with pair programming To assess student perception of the benefits or otherwise of pair programming, and issues related to its compliance and administration, an exit survey of the class was conducted. Questions in this survey were also answered using a 5-point Likert scale. The issues evaluated and the corresponding questions in the questionnaire are shown below in Table 3. Based on a survey done on 59 students belonging to three separate classes over two semesters, 24% of them admitted to splitting the work and doing their own share independently. This amounted to practicing cooperation rather than collaboration, which is the central to pair programming. There was also the tendency among some students (22%) to do the same work independently and submit the better one once again violating the principle of collaboration. These feedback data pointed to a degree of student resistance to full compliance with the pair programming model. With peer evaluation, although only 10% admitted they rated their partner too highly, this contrasted with our own observation of individual performance and the rating they received from their partners. We found that students tended to rate their partners highly or very highly (4 or 5 out of 5) quite consistently, even when it was the weaker partner.

TABLE III. STUDENT SURVEY ISSUES AND CORRESPONDING QUESTIONS Issues Questions Benefits Working in pair for doing programming assignments in class helped me get a better grade Experience I enjoyed pair programming There should be pair programming for programming assignments in other courses Working in a pair hindered my progress I prefer doing programming assignments in class on my own Opinion on My partner was cooperative partner My partner did his/her fair share of work I had the right partner Execution In my pair, we split the assignment exercises and worked on separate parts In my pair, we both worked on the same exercises and submitted the better of the two. Peer evaluation Opinion on the drivernavigator model Pair formation strategy In rating my partner I gave him/her more credit than deserved In rating my partner I gave him/her the credit he/she deserved It is possible to work collaboratively with one person as the driver and the other person as the navigator Students should be allowed to choose their own pair programming partner The professor should choose the partner for everyone Partners should be chosen randomly Not knowing anyone well enough in the class may be a problem in choosing your own partner Overall, student attitude to pair programming was positive. In terms of the benefit gained from pair programming, students thought it helped with their understanding of the assignments given (74%) and with improving their programming skill (76%). The use of pair programming in other future courses was supported by 71% of the respondents. Opinion was more divided on the driver-navigator model, with just 64% expressing their confidence in it. The method followed for forming pairs was viewed broadly negatively, with 72% expressing the opinion that students, not instructors, should choose their partners. Interestingly, although pairs were formed by instructors, only 10% thought they had the wrong partner and 59% expressed the opinion that not knowing others in the class well enough would make it difficult for them to select a partner. V. CONCLUSIONS Previous studies have highlighted the potential benefits of pair programming, which include - Better learning due to peer-tutoring Students feel more at ease, and thus can concentrate better on the subject matter, with a peer tutor rather than a professional teacher [14]. - Better utilization of time Students don t need to wait for instructor help if their partner can help. - Positive effects of legitimized collaboration Collaborative work, particularly during the school years, is treated with suspicion, and often regarded as cheating. Pair programming gives students early collaborative work experience, fostering the development of teamwork and communication skills that are highly valued in both the academia and the job market. - Reduced workload for instructors Pair programming reduces assessment workload significantly for instructors, leaving more time for creative efforts to improve teaching. Data on grades available to us so far, and as presented in this paper, suggest an overall improvement in student performance with pair programming. But students own opinion on this approach is somewhat mixed. There appears to be a certain level of reluctance among students to accept its beneficial effects. There is anecdotal evidence that this dissatisfaction with pair programming might be linked to students not being used to collaborative work in the past, and their natural resistance to being forced into a different mindset. Elements of the pair programming model that attempt to impose a structure on students work habit appear to be particularly unpopular the best example of which is

the requirement of switching driver-navigator roles at regular intervals. More than a third of the students did not view the driver-navigator-based model favorably. There also appears to be a strong feeling among students against the instructor forming student pairs. Instructor intervention only for students unable to form pairs by themselves may help achieve a higher student satisfaction level crucial for the successful administration of pair programming in the class room. Students learning experience in the introductory Computer Science course is crucial in forming their first impression of the discipline. Early success can lead to a lasting impact on student motivation. The findings of this study lend further credence to the strategy of using pair programming to help improve student performance and hence student retention and progression in Computer Science. But it also brings to light the existence of underlying student apathy to collaborative learning. A successful implementation of pair programming would depend on addressing administrative issues such as pair formation and assessment taking students concerns into account. [8] C. McDowell, L. Werner, H. Bullock, and J. Fernald, The effects of pair-programming on performance in an introductory programming course, Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education, pp. 38-52, February 27 March 3, Cincinnati, USA, 2002. [9] L. Williams, E. Wiebe, K.Yang, M. Ferzli and C. Miller, In support of pair programming in the introductory computer science course, Computer Science Education, Vol. 12, Issue 3, 2002, pp. 197-212. [10] B. Simon, and B. Hanks, First-year students impressions of pair programming in CS1, Journal on Educational Resources in Computing (JERIC), Vol. 7, Issue 4, pp. 1-28. 2008. [11] G. Braught, L. M.Eby, and T. Wahls, The effects of pairprogramming on individual programming skill, ACM SIGCSE Bulletin, Vol. 40, Issue 1, pp. 200-204, 2008. [12] J. Somervell, "Pair Programming: Not for Everyone?" 2006 International Conference on Frontiers in Education: Computer Science and Computer Engineering (FECS'06), Las Vegas, USA, June 26-29, 2006. [13] National Center for Women & Information Technology Resources website. [Online], Pair Programming-in-a-Box: The Power of Collaborative Learning, 2010, Available: http://www.ncwit.org/resources.res.box.pair.html [14] S. W. Ehly and S. C. Larsen, Peer Tutoring for Individualized Instruction, Boston: Allyn and Bacon, Inc., 1980. REFERENCES [1] E. W. G. Clua, A Game Oriented Approach for Teaching Computer Science, SBC 2008 Workshop on Computing Education, Belem do Para, Brazil, July 12-18, 2008. [2] H.Roumani, Design Guidelines for the lab Component of Objects-first CS1, Proc. Thirty-third SIGCSE Technical Symposium on Computer Science Education, D. Knox, Ed. ACM, New York, 2002, pp. 222 226. [3] L. Ray, S. Khan, W. Summers and B.Wright, Media Computation: An Audio-Visual Approach to Teaching Computer Programming, International Journal for Knowledge, Science and Technology, Vol.1, Issue 2, pp. 93-101, 2010. [4] M. Koski, J. Kurhila, and T. Pasanen, Why using robots to teach computer science can be successful theoretical reflection to andragogy and minimalism, Proc. 8th International Conference on Computing Education Research, Koli, Finland, November 13-16, 2008. [5] A. M. Phelps, C. A. Egert and J. D. Bayliss, "Games in the Classroom: Using Games as a Motivator for Studying Computing: Part 1, " IEEE Multimedia, Vol. 16, Issue 2, pp. 4-8, Apr.-June 2009. [6] L. Williams and R. R. Kessler, "Experimenting with Industry's 'Pair-Programming' Model in the Computer Science Classroom," Computer Science Education, Vol. 11, Issue 1, pp. 7-20, 2001. [7] J. Bevan, L. Werner,C. McDowell, Guidelines for the use of pair programming in a freshman programming class, Proceedings of the 15th Conference on Software Engineering Education and Training (CSEET'02), Covington, USA, February 25-27, 2002.