The Minneapolis Star Tribune has setup a nifty section of its website devoted to allowing users to evaluate and decide challenged ballots in the Minnesota U.S. Senate race -- they call it the "Ballot Challenge". This weekend, I dove in head first and took the challenge... More below the fold.
After evaluating over 5000 challenged ballots, I must say that a vast majority (considerably more than 50%) were indeed completely frivolous. In order to ensure the timeliness of the official review process and preserve the state canvassing board members' sanity, the numbers should be pared down significantly by the campaigns. The Franken camp has been doing just that, withdrawing 633 challenges on Dec. 3rd and 425 more challenges on Dec. 8th. Word via the Strib (Star Tribune) today is that by the time the state canvassing board reconvenes this Tuesday Dec. 16th, the number of remaining Franken challenges will be below 500:
Franken's campaign announced today that it expects the number of Franken-challenged ballots to drop below 500 by Tuesday. That's when the state Canvassing Board will start considering ballots challenged by campaign volunteers for Franken and Republican Sen. Norm Coleman.
Up to this point in the recount process, Coleman has followed Franken's initial move by withdrawing 650 challenges on Dec. 4th, an additional 475 on Dec. 9th, and 225 more on Dec. 11th. It remains to be seen whether Coleman will continue to follow Franken's leadership, as indicated by Al's intent to try cutting his stack down below 500 before Tuesday, to ensure that the challenges are both small in number and of a quality worthy of Minnesotan's patience.
Anyway, back to the Ballot Challenge. First off, a self-indulgent mini-rant. While the Star Tribune's interface is undoubtedly a great tool and a valuable public service, it could have been made a little more user friendly. Having to click twice (once to select your judgment on the challenge -- on a very small, easy to miss, radio button -- and another to "submit" your vote) to move from one challenge to another is a bit clunky. While it took me the better part of two days to evaluate all available challenges, simplifying the process to one click would speed the process up tremendously -- perhaps allowing users to dispense with all the challenges in a single day. When you’re dealing with thousands of ballots, every click counts!
Another issue I came across, the Ballot Challenge website stated that 6696 challenged ballots were available for viewing. However, the Minnesota Secretary of State's website said there were only 6655 original challenges. Unsure where the additional 41 ballots came from in the Strib's system.
One final set of comments before delving into the numbers. Note that these points represent significant caveats to my analysis. I was only able to make a judgment on 5381 ballots (out of the 6696 total ballots the website claimed to have online). Once I hit about the 5000th ballot mark, I started to notice a handful of broken ballots (ones that came up with no image and/or a missing full-ballot PDF link) cycling back through the process, even though I was clicking "skip" on those. After voting on ballot 5381, I received no new ballots at all. They were all completely broken after that point. Whatever happened to roughly 1300 additional ballots is beyond me. For one, I don't recall coming across the infamous "Lizard People" ballot. No idea what the overall distribution of missing ballots is/was or whether they would have favored Franken, Coleman, etc. Based upon the distribution of ballots up to that point, it is quite possible that they would have not significantly changed the numbers (unless they were certain challenges that had been backed out of the Strib's system?). I also have no idea what the breakdown was on how many Coleman vs. Franken challenges I saw. The underlying premise of these results is that the Strib's system presented ballots in an essentially randomized way such that the missing 1300 ballots would not have significantly impacted the trend up to that point in the analysis. I hope that is a sound assumption.
All that being said, take the numbers presented here with a heavy dose of caution. We just don't know how the other 1300 challenges would have fallen. Regardless of all this, the Franken team will continue to need our support through this process -- so don't even think about getting complacent or taking anything for granted based upon this measly little diary! All of this could be completely, totally, inexcusably wrong.
Now, on to the analysis...
First, concerning my ability to make ballot judgments. Full disclosure, I do have a political bias. I wanted Franken to win the election, believe that he did win the most votes (in an incredibly close contest, obviously), and sincerely hope the final recount tally reflects his victory. However, I am also a scientist. I am keenly aware of the importance of setting up conditions and evaluative methods that, by themselves, do not favor one outcome over another. So, starting from Minnesota Statute 204 C.22 (also summarized here), concerning determining voter intent, the guidelines I followed in allocating challenges were roughly:
- Stray, seemingly random, or incidental marks did not count toward voter intent or ballot disqualification. This class of marks includes smudges, pen bleed through, pen "trails", small pen resting dots, etc.
- Any significant marking (other than ones stated immediately above) in the voting oval counted toward the consideration of voter intent.
- Check marks, X marks, dash marks, slash marks, underlines, star marks, arrows, circles, large dots, squiggles, and enclosing marks made in a deliberate way counted toward the consideration of voter intent.
- In the event of an overvote, if the voter appeared to negate one of their choices with an X mark, slash, the words "no" or "yes", etc., that was considered to be enough information to validate a vote for one candidate.
- In the event of an apparent overvote, if the voter made a significant effort to indicate their preference for one candidate over another (outside the steps mentioned immediately above), that was considered to be enough information to validate a vote for one candidate (i.e., if a voter started a circle around the edge of a voting oval for one candidate but completely filled in the oval for a second candidate, that counted as a vote for the second candidate).
- Any non-overvote which included two or more types of markings to indicate the preference for a single candidate (i.e., a filled-in voting oval and an X mark, a filled-in oval and a check mark, etc.) counted as a vote for that candidate.
- Any mark not touching a voting oval and not clearly indicating a candidate preference did not count toward voter intent (i.e., the intent, if any, of a circle drawn completely between to voting ovals, without touching either, could not be determined).
- Any written message on the ballot was used, if necessary, to clarify the intent of a voter's markings (if any) or used to determine voter intent.
- If a voter selected a candidate and also used the write-in box to fill in the same candidate's name, it was counted as a single vote for that candidate.
- No write-in names counted as a disqualifying identifying mark.
- In the event that the intent of a voter's marking was in question, the nature of markings on the ballot as a whole was considered. For example, if the only markings on the ballot appeared to be similar to pen resting dots, and all those dots fell inside voting ovals, that was considered a significant enough mark to determine a voter's intent.
- Any immediately visible and obvious signature, address, initials, etc. of the voter disqualified the ballot. Note, in 99+ percent of cases, it was simply not feasible (in terms of time) to view the entire ballot. So, I suspect there were cases that a voter identified their ballot and I did not catch it.
I found that allocating challenges was pretty straightforward. The approach I used could be considered on the somewhat liberal (naturally!) side of decerning voter intent. There were certainly some head-scratcher cases. Many times I found myself pondering what the heck a particular voter must have been thinking. Do people not know they can get a new ballot if the first, second, N-th ballot gets botched up?! Anyway, here are the results of this unbelievably tedious exercise:
Federalist's Ballot Challenge Projection:
Total ballots examined: 5381/6696
Ballots allocated to Coleman: 2433
Ballots allocated to Franken: 2660
Ballots with votes for other or no candidate: 288
Projected Coleman vote total: 1211948
Projected Franken vote total: 1212036
Initial election result: Coleman by 192
Projected election result: Franken by 88
Projection error: +/- 36 votes
The Strib starts out assuming the margin is 192 votes in favor of Coleman (even though the latest number of the Secretary of State's office says Coleman by 188). Perhaps the difference could be explained by Minneapolis having counted that bag of about 20 uncounted absentee ballots found during the search for the missing "1/5" envelope containing 133 ward 3, precinct 1 ballots??? Regardless, going from Coleman +192 to Franken +88 indicates a net pickup of 280 votes for Franken by allocating challenged ballots and projecting the trend out to all challenged ballots.
I may be a scientist but I am not a statistician. Going on face value, and keeping all the caveats in mind, these numbers do seem to give credence to the claim made some time ago by the Franken team that they will be leading after the recount (their number is Franken +4). Any way you cut it, that's a heck of a good place to be leading up to the potential counting of wrongly rejected absentees!
So, that's all I got. Take it for what it's worth. Time for some sleep!
P.S. A little anecdote from going through all these ballots. One of my favorite ballots was one where the voter marked the write-in oval for President and wrote "PRESENT". I actually got a good laugh out of that. Those of you who get the "present" reference in terms of the Presidential campaign will probably be surprised that they voted for Al. Ironic, go figure. Al would probably find that pretty funny as well.
P.P.S. And, concerning the inevitable question about me and why I would spend my entire weekend pouring over scanned images of thousands of ballots, the answer is: no, I don't...but, I'm okay with that. ;)
Update [Mon Dec 15, 2008 at 03:04:00 AM CST by Federalist]:
From comments, users lompe and lenzy1000 are also well past 5000 ballots. Incredible work! Both evidently having better luck with Strib than I had. Here are their results thus far:
lompe's Ballot Challenge Projection:
Total ballots examined: 5700
Ballots allocated to Coleman: 2535
Ballots allocated to Franken: 2781
Ballots with votes for other or no candidate: 384
Projected election result: Franken by 54
lenzy1000's Ballot Challenge Projection:
Total ballots examined: 5100
Ballots allocated to Coleman: 2180
Ballots allocated to Franken: 2344
Ballots with votes for other or no candidate: 576
Projected election result: Franken by 22
Good stuff!
Update [Mon Dec 15, 2008 at 09:45:00 AM CST by Federalist]:
lompe keeps plugging away -- past 6000 ballots now. Franken +107. Nice work, lompe!
lompe's Ballot Challenge Projection:
Total ballots examined: 6000
Ballots allocated to Coleman: 2668
Ballots allocated to Franken: 2938
Ballots with votes for other or no candidate: 394
Projected election result: Franken by 107
Update [Mon Dec 15, 2008 at 01:13:00 PM CST by Federalist]:
lenzy1000 hits 5400.
lenzy1000's Ballot Challenge Projection:
Total ballots examined: 5400
Ballots allocated to Coleman: 2316
Ballots allocated to Franken: 2499
Ballots with votes for other or no candidate: 585
Projected election result: Franken by 45 (?)