| NSF Org: |
CNS Division Of Computer and Network Systems |
| Recipient: |
|
| Initial Amendment Date: | June 22, 2020 |
| Latest Amendment Date: | June 22, 2020 |
| Award Number: | 2033923 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Jeremy Epstein
jepstein@nsf.gov (703)292-8338 CNS Division Of Computer and Network Systems CSE Direct For Computer & Info Scie & Enginr |
| Start Date: | July 1, 2020 |
| End Date: | June 30, 2023 (Estimated) |
| Total Intended Award Amount: | $200,000.00 |
| Total Awarded Amount to Date: | $200,000.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
6100 MAIN ST Houston TX US 77005-1827 (713)348-4820 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
TX US 77005-1827 |
| Primary Place of Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | Special Projects - CNS |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
As a result of the COVID-19 virus, many states are looking to rapidly switch from in-person voting to postal voting (also called vote-by-mail or VBM), in order to reduce the infection risks to voters that might come from in-person voting. Among the many challenges of increased use of VBM, commercial ballot printing services have limited capacity and will be unable to scale to support the surge in demand they will face. To mitigate this problem, VotingWorks, a non-profit, is developing an open-source VBM solution called VxMail, targeting small- to medium-sized counties. VxMail will use commercial bulk-mailing services as well as off-the-shelf hardware for printing ballots, slicing envelopes, and scanning the ballots to provide a secure, affordable, and highly scalable VBM solution. If VxMail is successful, it will have a large impact on the practice of voting in the November 2020 general election, by making VBM available to more voters.
VotingWorks faces a variety of engineering challenges to ensure that VxMail will be accurate, including dealing with stray marks from voters, and damage that might occur to ballots in the postal mail. To address this, VotingWorks is partnering with a team from Rice University with experience in human-computer interaction and elections. This research will recruit thousands of participants, across the country, to fill out test ballots, yielding a large corpus of real-world examples. VotingWorks will use these ballots to validate and improve its scanning algorithms. Rice will use these ballots to measure voters? accuracy at filling out the ballots and following the return instructions. With multiple rounds of these experiments, Rice and VotingWorks will be able to identify and remedy deficiencies in VxMail, leading toward better voter experiences and more accurate tabulation results.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The research supported by this grant focused on two aspects of voting: 1) the usability of a new vote by mail ballot system, and 2) the usability of ranked choice paper ballots. The vote by mail portion of the research was conducted during the height of the COVID pandemic in order to better understand how to support a voting method that protected users from the virus. The ranked-choice voting study was conducted to gain a better understanding of a voting method which is gaining in popularity, with the hope that usability data could help guide the design of these ballots before they become ubiquitous.
In the vote by mail usability assessments, 28 participants from the general population evaluated a newly designed vote by mail system. Because of the pandemic, the study was conducted completely over zoom using a think aloud protocol. 23 of the participants were women, 96% had voted in at least one past election, and 54% had not previously voted by mail. Participants were asked to complete the mail in ballot using a slate of candidates that was provided to them. Following their completion of the task, they were administered the System Usability Scale (Brooke, 1996). While the System Usability Scores were relatively high (M = 79.7 (out of 100), SD = 21.02), only 25% of the participants were able to successfully complete the entire task. Critical issues that led to lack of success included failure of the voter to include a date on the ballot, failure of the voter to include a voter’s affidavit, voter using the incorrect return label, and failing to fill out the bubble ballot correctly. These failure modes indicate that the vote by mail design tested in this study will require additional design modifications and testing in order to bring its usability up to the standards that would be required for a mail-in ballot.
In the assessment of the usability of ranked-choice voting paper ballots, we examined three fundamental formats that are currently being used: the bubble grid format, column format, and handwritten ranking. Using a between-subjects design, 126 participants from the general population voted in a mock election using one of the three different ballot formats. The sample comprised a diverse sample of ethnicities and education levels. Approximately 90% of the participants had never used a ranked-choice voting ballot before. We collected information on the time to vote, the time to read the ballot instructions, the subjective usability of each ballot using the System Usability Scale, and errors committed while voting.
Results showed that participants spent significantly more time reading the instructions for the handwritten ranking ballot, but there was no difference between the bubble grid and column format. There was also a significant difference in the time it took to complete the ballot, with the handwritten ballot taking longer than the column format, but no other significant differences. Subjective usability scores using the System Usability Scale were quite high (with an average score > 80 out of 100) and roughly equivalent across ballot formats. Significant errors were committed by voters when using these ballots. 36% of the bubble grid format ballots, 14% of the column format ballots, and 2% of the handwritten ballots had at least one error that would have invalidated the ballot. 14% of the hand-written ballots, 12% of the bubble grid format ballots, and 12% of the column format ballots contained at least one mismark error that would not have invalidated the entire ballot, but might result in a selection that the voter did not intend. The next phase of the research will examine the electronic form of these three ranked-choice voting formats to determine if the computerized format has better usability characteristics than paper ballots, when common digital error correction/mitigation techniques are implemented.
Last Modified: 08/27/2023
Modified by: Philip T Kortum
Please report errors in award information by writing to: awardsearch@nsf.gov.
