Janice (Ginny) Redish Redish & Associates, Inc. ginny@redish.net Dana Chisnell, UsabilityWorks Ethan Newby, Newby Research (consultant on statistics) Sharon Laskowski, NIST Svetlana Lowry, NIST Center for Plain Language 2009 Symposium Washington, DC October 30, 2009 Redish, Chisnell, Laskowski, Lowry Slide 1
The National Institute of Standards and Technology helps implement the Help America Vote Act (HAVA) One facet: Research on instructions to voters 2005-2006, Ginny Redish reviewed ballots from every state and DC. That led to a report, guidance, and a paper. (References at the end of the talk) Redish, Chisnell, Laskowski, Lowry Slide 2
2006-2008, Ginny Redish and Dana Chisnell conducted an empirical study of language on ballots. 45 voters 3 locations Georgia, Maryland, Michigan suburban, city, small town 2 ballots that differed only in the wording of instructions on the ballot (and names of parties and candidates) Redish, Chisnell, Laskowski, Lowry Slide 3
Do voters vote more accurately on a ballot with plain language instructions than on a ballot with traditional instructions? Do voters recognize the difference in language between the two ballots? Do voters prefer one ballot over the other? Redish, Chisnell, Laskowski, Lowry Slide 4
Redish, Chisnell, Laskowski, Lowry Slide 5
Redish, Chisnell, Laskowski, Lowry Slide 6
A B Redish, Chisnell, Laskowski, Lowry Slide 7
Eligible: US citizens over 18 Focused on: Lower education levels Also balanced for gender ethnicity age (18-61; median = 36) Redish, Chisnell, Laskowski, Lowry Slide 8
Part 1 Part 3 Part 1 Greeting and informed consent Part 2 Voting each ballot (Counter-balanced; P1 voted A, B; P2 voted B, A; and so on.) Part 2 Part 3 Reviewing and choosing between ballots, page by page and overall Demographic and voting experience questionnaire Thanks and payment Redish, Chisnell, Laskowski, Lowry Slide 9
Ballot A Ballot B Total Correct 698 726 1424 Incorrect 112 84 196 Total 810 810 1620 45 participants; 18 possible correct votes on each ballot Marginally statistically significant, p<.071 Redish, Chisnell, Laskowski, Lowry Slide 10
Very little difference on B whether it was first or second. But A first, average correct = 14.4 A second, average correct = 16.3 Statistically significant, p<.001 Redish, Chisnell, Laskowski, Lowry Slide 11
"Keep" preferred by 83% "Keep" is short and sweet compared to "retain." "To keep." Yes. Yes. I do [want to keep her]. Like I'm thinking 30 seconds less. "For/against" preferred by 78% "For/against" are more common words than "accept/ reject." I prefer "for/against"; they are simpler words. Redish, Chisnell, Laskowski, Lowry Slide 12
Preference # of participants Ballot A 4 9% (traditional language) Ballot B 37 82% (plain language) No preference 4 9% Total 45 100% Statistically significant, p<.001 % of participants Redish, Chisnell, Laskowski, Lowry Slide 13
Education made a difference lower education correlated with more errors Situations with high error rates straight-party voting and reviewing party-based races to change some races with similar names but at different levels of government the screen at the end that shows undervoted races in red Redish, Chisnell, Laskowski, Lowry Slide 14
Redish, J. C., Chisnell, D. E., Newby, E., Laskowski, S. J., and Lowry. S. Z, Report of Findings: Use of Language in Ballot Instructions, report to NIST, December 2008, http://vote.nist.gov/nistir-7556.pdf (from http://vote.nist.gov/docmap.htm) Redish, J. C., Review of the Gap Between Instructions for Voting and Best Practice in Providing Instructions, report to NIST, 12/9/05, http://www.vote.nist.gov/instructiongap.pdf Redish, J. C., Guidelines for Writing Clear Instructions and Messages for Voters and Poll Workers, report to NIST, 2/21/06, http://www.vote.nist.gov/032906plainlanguagerpt.pdf Laskowski, S. and Redish, J. C., 2006, Making ballot language understandable to voters, Proceedings of the 2006 USENIX/ACCURATE Electronic Voting Technology Workshop (EVT '06), Vancouver, BC, http://www.usenix.org/events/evt06/tech/full_papers/laskowski/laskowski.pdf Redish, Chisnell, Laskowski, Lowry Slide 15
Ginny Redish, ginny@redish.net Dana Chisnell, dana@usabilityworks.net Sharon Laskowski, sharon.laskowski@nist.gov Svetlana Lowry, svetlana.lowry@nist.gov Redish, Chisnell, Laskowski, Lowry Slide 16