Administering Exams in Large Lecture Halls By Adrienne Williams November 2008 Have Left-handed or other Special Seating Students Contact You 1. Send an email requesting students reply if they need a left-handed seat. They MUST include their Lecture section (A, B, C, D, E) and their student ID number. You only need to do this for the first exam. 2. If you prefer, you can set up an EEE survey to collect this data (https://eee.uci.edu/toolbox/survey/library.php). This keeps your inbox clear and requires students to log in, give their section, and allows you to download the results in a spreadsheet-friendly format. 3. Require a response by three days prior to the first exam. Create a Seating Chart 1. This requires its own full explanation. A copy can be requested from adriw@uci.edu. 2. If the lecture hall is fully enrolled, call UCI Facilities at 824-5444 to request a few desk-chairs be brought in as temporary seating. Design an Appropriate Exam 1. Make it short enough to take in the time provided. This is particularly important for midterm exams, where there is at most 45 minutes available for test-taking. Assume 2-3 minutes for each multiple-choice, fill-in or matching question. To calculate the time for a free response section, have a grad student TA answer the problems and add 50% more time than it took the TA. It is best to not use discussion leader TAs to do this, as they become much more likely to (subconsciously, usually) teach to the exam. 2. Remember all images must photocopy clearly. Convert to greyscale before printing to help judge the quality. Photocopy a sample to double-check. 3. Minimize all of the above or c AND d types of answers on multiple choice questions. These become logic problems rather than biology problems. 4. The School of Biological Sciences asks that exams not be fully multiple choice if there are resources available for free response grading. If you have TAs, plan your free response section so that each TA can finish grading in 10 hours or less. 5. Your exam will be VASTLY improved if you allow your TAs to take it and give you feedback before the final version is printed. Again, not discussion leaders if possible.
Create a Second (and Third) Version of the Exam 1. Make the first question of the multiple choice exam the version question. It should read: 1. You have Version A. Please bubble in a for answer 1. 2. At minimum, create a second version with the questions in the same order but the answers rearranged. 3. If you want to scramble the question order, scramble version A first, and then scramble it a second time to create version B. People often write questions in the order in which the material appeared in lecture and use that as version A. There is anecdotal evidence that students who take this version A have a slight edge over the ones who take version B. "Double scrambling" makes it fair for everyone. 4. If the question contains a figure that makes it recognizable from another seat, consider making a different version of the question that asks a different question about the figure. If you can t do that, at least move the figure and the answer blank around so that something has changed and might discourage copying. 5. If a short-answer question will produce a figure or other large visual answer, be sure different exam versions will produce a different visual answer. Design the Exam for Rapid Return 1. Read more about Rapid Return here: https://scandocs.ddm.uci.edu/app/rris/default.aspx 2. Plan on using Rapid Return to give a pdf image of the scantron itself back to the students 3. If you have a free-response section, it is cheaper to send only this to Rapid Return rather than the whole exam. Create a second booklet of just the free-response questions, and make the front page the cover sheet generated by Rapid Return. 4. Create 2-3 versions of the multiple choice booklet, and call these versions 1, 2, and 3 Photocopy the multiple choice booklet on lightly colored paper. 5. Create 2-3 versions of the free response booklet, and call these versions A, B and C. Photocopy these on white paper. 6. Now you can tell students You should each have a yellow booklet with a scantron inside and a white free response booklet. Multiple Choice Booklet Free Response Booklet You have VERSION 2 Your scantron is inside. Fill it out now, but do not begin the exam. Scantron Form You have
Generate Rapid Return Cover Sheets 1. Contact your department bookkeeper to find the name of the appropriate bookkeeper for this class, and the account and fund the scanning will charge to. 2. Go to the Rapid Return section of UCI s Distribution and Document Management web page: https://scandocs.ddm.uci.edu/app/rris/default.aspx 3. Log in to access the cover sheet page 4. Enter the appropriate information, including the bookkeeper UCInetID and the account and fund for the course 5. Print the resulting cover sheets. Save them as pdfs in case the printout is lost. 6. Attach the cover sheet as the first page of the free response booklet. Photocopy the Exam 1. Do not photocopy different versions different colors. While this would make it easier to see that students are seated next to different versions and sort for grading, it also makes it easier for students to choose whom to cheat off of. 2. As above, photocopy the multiple choice booklet on colored paper and the free response booklet on white paper. 3. Have the exam booklets photocopied doublesided. 4. Order scantron forms along with the exam 5. Order 5-10% more exams than there are students. Organize Exam Support Staff 1. The School of Biological Sciences requires six proctors for large lecture exams (though they don t state what large is). A minimum is three proctors for 200 students is best. 2. You will need at least three people per lecture to assemble the exams after photocopying 3. Decide whom you will use for these jobs. When emailing to arrange participation, be sure to ask for participants to reply back, rather than just hoping someone will show up. Posting the Seating Chart 1. A chart of student names and assigned seats may be posted to the class webpage no earlier than 1pm and no later than 10pm on the day before the exam. Do not use student IDs as identifiers. 2. Also post (or email) exam rules, for example: nothing at their desk, all materials up front, no hats, no cells phones, only capped water bottles, no electronic devises, no hoods up on sweatshirts, only # 2 pencils on scantrons etc.
3. Paper copies of the seating chart should be posted on the doors of the lecture hall 20-30 minutes before the exam. Instruct a TA to remove the paper version after the exam. 4. Have a spare copy of the seating chart on the table at the front of the lecture hall. Assemble the Exam for Quick Disbursement 1. Assembly for each large lecture takes about 30-45 minutes with 3 workers 2. Insert a scantron form inside each multiple choice booklet. 3. Collate multiple choice booklets (e.g., 123 123 123) 4. Make passing out the exams proceed very quickly by bundling exams by the number of seats in each row. Go to http://www.reg.uci.edu/etc/campus/ for a list of campus classrooms. Click on your lecture hall for an image that shows the number of seats per row. Create a permanent document that shows the rows for each seat section (right, middle, left) and the number of seats in each row. See PSLH sample: Row Even seats (Left Side) Middle seats A 3 7 3 B 6 8 6 C 7 8 7 D 8 9 8 E 9 10 9 F 10 11 10 G 11 11 11 H 12 12 12 J 13 13 13 K 14 13 14 L 14 14 14 M 14 14 14 N 14 14 14 O 10 14 10 Odd seats (Right Side) 5. Count out batches of multiple choice booklets for each row. Write a small row letter on the top booklets for each row. Stack the booklet batches according to section, and mark each stack (right side/odds, middle/100s, left side/evens). This will allow you to pass out the booklets very quickly and accurately. 6. Collate the free response booklets (e.g., 123 123 123) 7. Count out the free response booklets to match rows, as in #5 above.
8. Place exam stacks back into boxes. Make sure all stacks are clearly marked. Make sure all BOXES are marked with the correct lecture. Materials to Take to the Exam 1. The exams 2. Five extra seating charts (if emailed / posted the night before). Four to post on each entrance to the lecture hall, one to leave on the front table. 3. A seating chart sorted by seat number rather than by name, to check attendance 4. 10 decent pencils with erasers to loan students 5. Stapler 6. Package of tissues 7. Clipboard with notepaper and pen. 8. An explanatory PowerPoint that shows: a. A map of the lecture hall (e.g., where the even and odd sides of the room are), what to keep at your seat (ID and pencil only, usually), turn off and put away phones, and that students must sit in their assigned seat. b. How to fill out a scantron c. How to fill out a Rapid Return cover sheet To Announce while Students are Seating 1. An extra seating chart is posted on the table at the front of the room 2. Remember to place all items at the front 3. Sit in your assigned seat. If someone is in your seat, come talk to the coordinator Passing Out Exams 1. Pass out multiple choice booklets first. 2. Let 30-60 seconds lag before passing out short answer booklets. 3. Tell students to take the top exam in the stack and pass the rest on. 4. If the class is full, you can tell students to leave an exam on the empty seats next to them. To Announce to Begin the Exam 1. All students should have a yellow (or whatever) multiple choice booklet and a white short answer booklet. Turn in everything when finished. 2. Turn caps around, hoods down, make sure eyes and ears are visible
3. We will not answer questions during the exam. Only raise your hand if there is a problem with your exam booklet. 4. You may write on the booklets, but only marks on the free response booklet will be graded. 5. We will post the time on the screen (use www.time.gov). You may not use your phone as a clock. 6. Keep your exams and scantrons down on your desk. Cover your answers as much as you can. Keep your eyes down or straight in front. 7. Check your phone AGAIN. We will keep phones we see. 8. You may turn in your exam and leave when you are done. Instructions to Proctors 1. Stay in front and to the sides of students. Don t stand above and look down 2. Don t look over the exam watch student eyes 3. If a student s eyes wander, tell the coordinator / faculty right away. Don t stand and stare for 15 minutes, wondering. 4. Don t answer student questions about content. Just say give your best answer. 5. When you collect exams, make sure they turn in both booklets and the scantron. Quickly check that the Student ID number is both bubbled in and written in. The number should start at the left end with two boxes empty at the right end. Check the first 4 digits against the Student ID, and make sure the photo is at least similar to the person turning in the exam. 6. Put multiple choice booklets into a box. Sort short answer booklets by version into another box. Sort scantrons into ten stacks based on first digit of student ID number if time permits. During the Exam 1. Train 2-3 proctors to seat latecomers. Place them at the top and bottom of the lecture hall. If the lecture hall is full and you ve left exams in empty seats, the TAs can escort latecomers to their assigned seats. 2. If you did not have exams left in empty seats, put latecomers on edges or in front rows so you can give them the appropriate exam versions. I try to save the front row and extra left handed seats to give to latecomers. 3. If you or the proctors see anyone who even MIGHT be looking at another exam, quietly move them to a front or end seat. In the rare event that the student asks why? you can say that there were people looking around near them. 4. Do not answer student questions about content.
If the Exam Runs Long 1. Promise yourself (and your proctors) your next exam will have fewer questions. There just isn t any good way to safely collect exams in this situation. 2. Warn students they have X minutes to turn in their exam on their own. At Y o clock, they will have to turn over their exam and put their pencils down and wait for you to collect. 3. Announce penalties. Any students caught working past this time will have 5 points docked automatically from their exam. Any student who has not written in and bubbled in their ID number will have 2 points docked. 4. As Y o clock approaches, send out proctors to watch students. 5. Announce that time is up, and all exam materials need to be turned over. Anyone caught writing will have their exam marked for point removal. 6. Have all scantrons sent to the aisle. Collect them. 7. Have all booklets sent to the aisle. Collect them. 8. Release students. Grading Scantrons 1. Requires its own explanation. Email Adrienne at adriw@uci.edu for a copy. Grading Free Response Questions 1. Reserve a room with good chairs and tables for graders. 2. Faculty member should provide cash for grader food plan on about $8 per grader. 3. Order food, and turn in receipts to the department for reimbursement from course funds. The department bookkeeper has a form to be signed by the course instructor. 4. Faculty should have three copies of the answer key for the free response questions ready immediately after the exam. 5. Assign either 1 TA to each page, or a team of 2 TAs to each page so they can compare scores and get feedback. Set up 20-30 exams to be open to the appropriate page, and skim the answers and compare with the key. 6. Faculty should stay with the graders for the first 20-30 exams to get an idea of the range of answers and how to award points for partially correct answers. 7. Graders should maintain a key page that indicates acceptable answers and how points are awarded. 8. Every grader should write the points earned for EACH question to the right of each question (+1, +2, etc). Use a dark red or green or colored pen. Avoid blue or black, so that student marks are separate from grader marks. 9. Mark wrong answers with a +0.
10. Graders should sign their initials at the bottom of the page, and write the total number correct. 11. Give each grader a blank exam so they can write out the most common wrong answers, and give feedback about how to improve the question for future exams. 12. As graders finish grading their questions, they should start adding up scores and writing these on the front page, and sorting the exams by student ID number. If time permits, two different people should add up the free response points for a more accurate score. 13. Enter the free response scores into a spreadsheet next to the scantron results. Have another grader check the entries. 14. Go to the Rapid Return website (or call) to arrange pickup of scantrons and short answer booklets. Creating the Key 1. After the scantrons have been run, check the Webtron results to be sure the key was accurate. 2. If any multiple choice questions received another answer as often as the correct answer, check to be sure that the second answer isn t also valid. Re-run through the online Webtron with modified key if necessary. 3. Collect the accepted answers from the TAs for the free response questions. 4. Create an exam key that has the correct multiple choice answers marked, and an explanation of points for each free response question. Make this a pdf. 5. Post the exam key on the class web page before releasing grades. This allows students the opportunity to suggest errors in the exam or key before grades are released. Releasing Grades 6. Wait for the Rapid Return files to be uploaded to the student dropboxes on EEE before releasing grades. This can take up to 72 hours, so you may want to warn your students. 7. Upload number grades (scantron + free response) to the class Gradebook. Do not release these grades until the dropbox has the Rapid Return files for students to see. 8. Never give students their grade by any method other than Gradebook (this maintains student privacy). 9. When the Rapid Return dropboxes contain the student free response booklet and scantron pdfs, students and you will receive an email. Now release the grades in Gradebook. Send an email reminding students that the key is on the class web page, and of the regrade procedure.
Regrades 1. Make sure the regrade procedure is posted as part of the class syllabus 2. Create an online EEE survey for students who want to request a regrade. This cuts down on emails from students saying, please regrade my whole exam or I should have two more points somewhere. Ask for: a. Lecture Section (A, B, C, etc) b. Question that was graded incorrectly c. Reason it was graded incorrectly 3. Do not accept requests after the deadline (about one week is adequate).
Exam Proctor Instructions (To be given to proctors before exam) Arrive early 1. Arrive at the lecture hall 10 minutes before the exam begins 2. Help seat students, have them move backpacks and notes to front of the room. Keep working area clear. 3. If you are not needed to help students, start counting out exams into small stacks with the right number of exams for row A evens, row B evens, etc. Pass out exams 1. If the exams are not pre-stacked by row, they will need to be passed out quickly. Some TAs should estimate the right size of stack to hand to the first person in the row. Other proctors should be on the other side of the row to pick up extras or pass out if the stack was short. 2. If you see papers or bags at students seats, make them take stuff up front (or take it yourself). Seat latecomers 1. If an exam was placed at every seat, escort them to their seat 2. If no exam is at their empty seat, put them in an empty seat (usually front row or extra left-handed seats) with an appropriate exam version. Tell coordinator where the new seat is. Watch for cheating 1. Watch students from the side and below, so you can see their eyes. Don t proctor from the top of the room 2. Don t read over the exam while proctoring 3. If you see a student s eyes wander once, watch longer. If they wander again, tell the faculty member or coordinator to get them moved. Staring isn t stopping them. 4. Don t chat with other proctors Collect exams 1. Stand back from the front row while accepting exams so lower students can concentrate 2. Take all booklets and scantron. Make sure scantron has ID numbers and bubbles filled in. Tell student to fill these in if they aren t done. 3. Take student ID. Check: a. First four numbers are same as on exam b. Name is the same c. Picture looks at least somewhat like the student 4. Give the ID back with a thank you and a smile. Sort and pack exams 1. Place multiple choice booklets (if separate) in one box 2. Sort free response booklets by version. 3. If time, sort scantrons by first digit of student ID. If the exam had a version question, the exams do NOT need to be sorted by version.