More from the Interview with Beth Simon on Peer Instruction Interviewer: Beth Quinn, NCWTI Research Scientist & Director of EngageCSEdu HOW BETH SIMON GOT INTO PEER INSTRUCTION AND GETTING OTHERS TO TRY IT 1 DOES IT WORK? HOW DO WE KNOW? 2 CLICKERS PROVIDE KEY DATA 3 USING PEER INSTRUCTION IN DIFFERENT CONTENT AREAS AND CLASS SIZES 3 HOW BETH PULLED OFF PEER INSTRUCTION IN A REALLY BIG CLASS 4 GETTING STUDENTS ON BOARD 4 PEER INSTRUCTION AND BUILDING STUDENT COMMUNITY 5 MORE ON FORMING DISCUSSION GROUPS 6 WHY WOMEN-ONLY GROUPS? A NOTE FROM THE INTERVIEWER 6 How Beth Simon got into Peer Instruction and getting others to try it Beth Quinn (BQ): You, and your colleagues at UCSD, are well-known for using, and researching, peer instruction in your classrooms. How did you come to use this technique? Beth Simon (BS): My PhD is in optimizing compilers and architecture. So, I spend a lot of time thinking about how to modify things so they run faster. Where is the low hanging fruit that s going to get me the biggest gain for the least amount of work? When I finished my PhD program, I chose to go to a teaching position and I started thinking about my teaching in the same way as my work in supercomputing. That is, how do I know that it s working and what is the most important thing for me to do to better improve the learning experience?
Then I got introduced to Richard Anderson at the University of Washington and I was invited to an NSF workshop hosted by Sally Fincher and Josh Tenenberg that took computer science professors and taught them about education. That changed my life! But the kicker that really led me to peer instruction was a year working with Carl Weiman at the Science Education Initiative at the University of British Columbia as a Science Teaching and Learning Fellow. I called it my own personal postdoc even though I was already a faculty member. I really was able to go all in on reading the education research. What that experience gave me was not only the education research chops but also the confidence. Nothing like working with a Nobel Prize laureate to give you confidence. So those two things, the two-year program in CS Education Research and then working with Carl Weiman absolutely were the genesis. Peer instruction comes from physics education research, especially Eric Mazur s work. When I saw how it was actually done in physics, I thought, this is perfect for computer science! Physics instructors are worried that their students were turning the crank on their equations but not really understanding what was going on. My concern with introductory programming is that students are writing programs where they encountered a number of bugs or problems along the way. They may have gotten their way through, but I don t know what they learned from that. But peer instruction can be scary for people trying it for the first time. I made mistakes at first: I didn t assign seats, didn t assign groups, I didn t make sure girls sat together, I forgot that I wasn t supposed to show the graph after the first vote. But if you aren t where a colleague can help you, simply shoot some video of you using peer instruction and send it to somebody like me to get feedback. It s interesting. I know two groups of people who sort of have indicated to me through -- we call it my diaspora -- have taken up peer instruction. Most of them have been graduate students from UCSD who then, again, worked with me in some way and then went out various places. And then there are a couple of faculty in my own department who usually didn t even tell me, only one even asked for any help. Especially if they adopted questions from my graduate students--which our department chair did. He s computer architecture and he trusted all this. But the others that are interesting to me are like my research colleagues, like Mark Guzdial and Robert McCartney, and they re like, We re thinking about trying it. And I say, Why did you do it? And they re like, Well, you keep presenting all these results! Which is interesting to me because that s the same way that I started doing pair programming. Brian Hanks was giving a talk and I m like, So remind me, like this is the THIRD study about this, right? So how many more studies about this do you have to do before I decide to adopt this in my classroom? And he s like, I don t know! Does it work? How do we know? BS: I have a freshman seminar on teaching computer science, and a student asked me, Well, how do you know it was the peer instruction and not the attendance, because they have to be in class. I said, I don t know, and I can t tease that apart. And to some extent, I m actually not interested in teasing that apart because what you don t know--you think attendance--but there could actually be some N other educational psychology things that play into this: growth mindset, retrieval practice, community building, all of this sort of stuff. I d never be able to tease all of those things out. But the number of different
educational psychology and learning theories that you can see contributing to that makes it really clear to me. I d rather doubt that Eric Mazur knew all that stuff going up front. And I m always surprised that there s not some research study trying to at least, in some theoretical fashion, evaluate it from all of those components. But to me it s very clear as I teach more classes about teaching that that must be the secret sauce. Clickers provide key data BS: We were getting ready to run the CS principles pilots and we were at a big meeting. Jan Cuny calls me into the hallway and says, Beth, I m really concerned because you re piloting this course with 350 people and I feel really uncomfortable about that. First off, do you not understand the definition of pilot? I was like, Well, I have to teach 350 because I have to teach all of the sections, so I just want to do it all at once. I think it s more efficient. She responded, Well, could you run one section and leave the others? I said, Well, how big of an experimental class can I run so it will be good? And she goes, 70! And I m like, 70? How do you know that? She s like, That s where you can still look em in the eyes. I said, Jan, I understand that. But I don t have to read their minds, I have their votes. How amazing is it that Jan just responded, You re right, Beth. Okay. So, we did it with all 350. That was really pivotal to me in terms of explaining it to faculty. I think it really resonates with how faculty--especially in large classes--feel: Why can t I get them to tell me? Well, clickers just give you that insight. With computer science faculty, I ask them, Why are you trying to optimize your teaching with no data? Clickers makes it less of a mystery. You know, most faculty who have poor teaching evaluations find it frustrating because they don t have the data they need to make an improvement. It all comes down to the fact that faculty often don t understand why students won t ask questions, and what students do and don t know. They also feel like there s an adversarial relationship with their students. And I m like, We can change that. Using Peer Instruction in Different Content Areas and Class Sizes BQ: Can you use this technique in any class size? BS: I ve done it in classrooms as small as 20, I ve done it with 50, and I ve done it with over 400. The largest class I ve done this in is 465 students and they were spread out across three different rooms! We don t have one classroom big enough for that many students at UCSD. John Glick at University of San Diego uses it in all of his classes which are 5 to 12 students. He uses it a bit differently. If even one student gets it wrong, he can say, Well we didn t all get it so let s discuss. He says that works well in their atmosphere. BQ: I used to use clickers in my courses. Because I was teaching sociology of law and inequality, a lot of it was related to political or social issues. Sometimes it was difficult to get students to speak, especially in the larger classes. But when I started using clickers in those large classes (some as large as 275) an interesting thing happened. When students saw the quantitative results of a question, and it showed than
even a few other students answered the same way as them, it emboldened them to speak. They knew there were people out there who agreed with them. BS: That s interesting especially because I advise the UCSD history department in adopting peer instruction. In a computer science class, the skill I wanted to develop is analysis of detail, and logic-ing it through and thinking about what will happen. But the skill the history department wants to build is argumentation. So, we wrote questions where an argument could be made for every answer. I think that s interesting. Now that I m teaching in Education, I write more of those kinds of questions. It s really powerful, especially in sensitive topic areas where, you know, you re not saying, Oh, if somebody feels this way. I m saying, even if nobody feels this way, I m hoping that somebody makes an argument for it. Your job in your group is to discuss all the answers. So that s really cool. How Beth pulled off Peer Instruction in a really big class BS: At UCSD we have these two big lecture halls that are like 200-ish right next to each other, and there s kind of a little airlock in-between. Then, there is one behind with 25 seats. So, we used all three rooms and they live simulcast me onto big screens--i was larger than life--in addition to the slides. I use about one teaching assistant per aisle. Then we have this fabulous wireless microphone set and we got a doorbell system meant for a mansion, big mansions with six different doorbells! They have the stables, the pool, whatever! So, you tape the doorbell to the microphone and the doorbell thing has visuals on it. So, you turn off the ringer and you have your TA sit there and watch. And when they re all out doing the discussion you have TAs running around with the microphones in all these rooms listening in on the students discussion and they find students. They say, That s a really good point. We need to share that. Here s the microphone. And I have them, they have a bag of candy, Here s your piece of candy. Because if you talk in a class with 465 people, you deserve a treat. That s putting yourself out there. So, we use this method of the TAs sort of pre-identifying them and authorizing them, saying, That s really important. Even if it s not the right answer everybody needs to hear that misconception because common. So, the students feel better about that. And then they would ring the doorbell and my TA is writing down numbers, she s like, Number two. Number four. And I m in one room, there s three other rooms--i announce, Microphone number 4, you had something to say. And they d go, Hello? It s actually really great because nobody even looks. You know? It s not embarrassing because nobody knows where the voice is coming from. So, everybody just faces forward and listens. I just love that. I ve never even written about that or I m not even sure I ve talked to anyone about it. I think I might have told Carl that I ve done it. Getting students on board BQ: Do you get push back from students? BS: Sure. But you need to explain to your students why you are doing this. Repeatedly! You ll have these high performing students who have techniques they use to get A s in classes and you re changing the game on them. There can be a lot of angst. But my favorite quote from one of my evaluations went something like this: Dr. Simon was clearly very passionate about computer science education and she s
really good at explaining. I just wish she d stood up in front and lectured because I had to learn it on my own. I m like, Ding! Ding! Ding! BQ: Are there some things that instructors should avoid? BS: We ve studied variations in implementation and identified a few things that people did differently where it didn t seem to matter to student preferences or evaluations, and a couple of things that seemed to tank it. The main thing that will tank things is requiring students to get the answer to the clicker questions correct. Then it just becomes this big testing environment. People do that because they worry that students won t take it seriously unless it s graded. But that s the wrong direction. I compare it to sports. Cynthia Bailey Lee found this fabulous picture of an old-style aerobics class that we use. I say to students, If you were trying to get more fit, would you go sit and watch an aerobics instructor? No! That would be ludicrous. We talk about how cramming doesn't work anyway. If you were on the track team, would you not train all season and the just train the night before the meet? No. You wouldn t do that. Okay, so it s the same thing here. A lot of our kids have a lot of experience with either being coached music or sports. So, the coaching metaphor wins them over pretty well. I also talk with them about the fact that retrieving your knowledge is what really causes you to learn, and your brain is never more primed to learn than when you ve just tested yourself. Well, you only get tested in some classes twice! Midterm and final, when it s too late. So, say to students, I m building this testing into so your time with me is more efficient. You literally just tell them all the things. I have a slide deck that you can use, and you can walk students through them, explaining why you are doing peer instruction. But usually the problem is this: faculty stew about it for a long time and then they finally go, I m doing it! They really get bought in, but they completely forget to get the students on board. You have to build that in almost every week. Then the most important thing is to include quotes from students. It s better if they re quotes from students at your own institution but if not, you say, Yeah, we haven t done it yet here in CS yet but at the University of California they did and here s what some of their students said. When I ve done peer instruction with really big classes you end up with some really erudite students who say amazing things. They re like, Oh my god! This is the best class I ve ever had. I didn t have to study at ALL for the midterm because I learned everything I needed in the classroom. Boy that s a good one! Peer Instruction and building student community Another interesting thing I ve observed: Usually around week three or four we hit the first question where everybody gets it right after the group discussion. And they BURST into applause for themselves. You know? Four hundred people clapping! BQ: That s a great sign of an emerging collaborative--rather than competitive--student environment. They re applauding each other! They re seeing how everyone is coming along. Right there, that s a perfect a moment where you have student community.
BS: YES. I have quotes from 400-person class: Despite the fact that there were 400 people in this class, this was the smallest class I ever took. More on forming discussion groups BS: The group size is 3 because in a lecture hall that s who can face each other. You don t have to do any crunchy things, turning around or whatnot. I m in California so I can t get information on ethnicity, otherwise I would do that. I ve seen situations where we ve had foreign exchange students coming in and they were all speaking Italian, and I m like, I will put them all together. That s fine. My goal is for them to learn computer science not English. So, I believe that it s very important that you have assigned seats. And the funny thing is--you know I ve got this big class roster, 200 people, I just sort by gender, and I assign the seats. So, there s this section of girls. And usually it s about week 7 that somebody pulls me over and says, Did you notice how the girls are all sitting over there? And I m like, Oh yeah, that s interesting. Why Women-Only Groups? A Note from the Interviewer Some readers have asked for more information on why it may be important in the early courses to form women-only groups. Both Beth and I have observed that in large introductory computing classes, some men may avoid talking to women, or alternatively, may dominate the conversation. Either experience is problematic both for learning and in terms of women s sense of belonging. Why is it especially important in early courses? Many of these students don t have as much invested in the major so it s easier for them to switch majors if they have a bad experience. By making all women groups, you may avoid some of these dynamics. But once women have experience in the major, the assumption is this will dynamic will lessen, or at least, the women will be committed enough to the major that a few bad experiences won t crush their interest. And at that point, women and men need to be working together because, we hope, that will be their experience in the workforce in the future! To learn more about Beth s work with Peer Instruction, go to http://www.peerinstruction4cs.org To learn more about collaborative learning and broadening participation in computing, check out https://www.engage-csedu.org/engagement/grow-positive-student-community/collaborativelearning