Diary after diary, blog post after blog post, newspaper story after newspaper story ... they all report that Norm Coleman's lead in the MN-Sen recount is shrinking to 185 or 173 or 172 or some other tiny number.
They're all wrong.
In reality, as shown by the Minnesota SoS, as of last night Coleman's lead had shrunk by 43 votes, from 14,758 to 14,715.
Those other reports all add the original-count results from precincts that haven't yet reported their recount outcome into their numbers. But there is absolutely no justification for doing that; as far as the recount is concerned, those precincts don't exist yet.
As the dKos chronicler of the 2004 WA-Gov recount, I believe I have some credibility in such matters. The discussion continues below the fold...
As I indicated in a couple of widely-unread comments, on one of last night's recount diaries (here) and Markos's FP story (here), it's entirely appropriate to compare the outcome of recounted precincts to the original counts in those same precincts. But it is illogical and misleading to extrapolate an artificial overall margin for the full state, particularly when it's done via the simplistic inclusion of the original counts in not-yet-recounted precincts. Sure, doing this makes for exciting reading (and ghopod knows that newspapers, cable news, and such love fluffing their BREAKING NEWS!!!!). But it's not in any way reality-based, not in any way analytical, not in any way valuable.
As it happens, the first day of recounting in Minnesota tallied precincts that were relatively favorable for Norm Coleman in the original count. According to the Minnesota SoS, Senator Coleman originally received 43.3% of the votes in the recounted precincts while Al Franken got 40.0%; overall, each of them had 42.0% in the original statewide count. In the recounted precincts, Coleman originally led Franken by 14,758 votes (195,708 to 180,950). In the recount, Franken volunteers challenged 106 ballots while 115 ballots were challenged by Coleman workers, so it comes as no surprise that the vote totals for both candidates decreased slightly, to 195,638 and 180,923, respectively. With Coleman down by 70 and Franken by 27, the actual recount margin after the first day of the recount came to 14,715 (14,758 - 43).
That the margin became ever-so-slightly smaller is, of course, a good thing for Al Franken. It's perfectly OK that Coleman leads by about three percent in the precincts that have been recounted, especially when we see that his margin in these Coleman-preferring precincts has decreased a little bit. My point is that the infinitesimal margin, in the vicinity of 200, that's "reported" in the traditional and new media is simply not correct and not logical. It doesn't make logical sense, and it's not the way I reported the results of WA-Gov four years ago. Why, even as late as December 18, 2004, Dino Rossi led Chris Gregoire in the recount by almost 155,000 votes. Then the thirty-ninth and last county (King, of course) reported its recount numbers, and Gregoire was the winner!
In looking at the precinct-by-precinct spreadsheet available for download from the Minnesota SoS, I observed a column titled "TotalVotersInPrecinct", which I took to represent the number of valid ballots recorded in the original count. That number will not necessarily match the number of valid ballots in the recount, because the tabulators may have misclassified some ballots, undervotes and overvotes may be resolved differently in the recount, and the campaign workers may challenge ballots. Because of the challenges, we cannot yet ascertain the total number of valid ballots in a precinct, but it is possible to make an estimate that will always be greater than or equal to that number:
recColeman + recFranken + recOther + challbyColeman + challbyFranken
In that sum, "rec" indicates the recounted number and "challby" is ballots challenged by the named campaign. The three "rec" values are now fixed minimum counts for those candidates, whereas the "challby" numbers will eventually be adjudicated by the statewide Canvassing Board and either distributed into the three "rec" candidate totals or removed as invalid ballots.
Summing across all 957 precincts counted as of last night, the total number of voters in last night's total came to 452,249 (by subtraction, we see that 75,591 of them cast their votes for Barkley or another minor candidate). Using the equation value shown above, I determined an overestimate of 452,367 votes in the recount -- 118 more than in the original count. Assuming that all of the challenged ballots had previously been assigned to a specific candidate, and therefore had been included in one of the three original candidate categories, these extra 118 votes represent the minimum number of ballots that had been rejected by the tabulating machines but could be ascertained by humans to have been intended to go to a particular candidate. Once again, we see that humans are better at pattern recognition than are machines.
In one of my WA-Gov reports four years ago, I estimated that our hand recount produced 4.13 new-found votes per 10000 ballots. A similar calculation can't yet be made for Minnesota, but if my assumptions are correct the equivalent quantity for Day 1 of their Senatorial recount is, at minimum, 2.61 per 10000.
I hesitate to suggest that someone among our Minnesota Kossacks undertake to produce a "MN-Sen series" akin to the 28 diaries I wrote between November 8 and December 29, 2004. But I'd be extremely grateful were someone to present to our community a comprehensive, logical, and accurate analytic report on the progress of this recount. Alas, what I've observed in the last several days (here, on Nate's site, and in the tradmed) falls far short of what I'd hoped to see.