(This is an excerpt from a post to my website, Malark-O-Meter, which statistically analyzes fact checker rulings to make judgments about the factuality of politicians, and to measure our uncertainty in those judgments. My first analysis found statistically significant differences in the amount of malarkey spewed by Obama/Biden vs. Romney/Ryan, which would please the progressive DailyKos audience.)
The era of rapid fact checking is upon us. On an almost daily basis, websites like PolitiFact.com, FactCheck.org, and The Fact Checker give us in-depth analysis of the facts, and how they compare to what politicians say. PolitiFact and The Fact Checker go two steps further by using categories to rank factuality on an intuitive scale while maintaining up-to-date report cards on individuals who have been fact checked.
Categorical ranking systems make it easier to internalize and remember the results of a detailed fact check. Together with individual report cards, the categories give us a sense of someone's overall factuality. Yet these report cards are only a small sample of the statements that individuals (or groups) make. Furthermore, a list of counts in different fact checking categories provides no simple, singular measure of factuality that most people can easily interpret.
I created Malark-O-Meter to solve these two problems by using sophisticated statistical and computational methods, whereby I can make inferences about an individual's factuality from a small sample of statements. Beyond the measurement of factuality, I hope to convince people that they must consider the certainty with which they can make statements about the relative truthfulness of different people, especially political opponents. As you'll see, my analyses also belie the hyperbole spoken by one side against the other.
Malark-O-Meter starts with a simple scoring system. I assign numeric values to each category in a ranking system, with more false statements receiving higher values. Then I multiply those values by the percentage of statements made in that category. Finally, I sum the results. I call the end product "bullpucky". A bullpucky of zero means you are always factual. A bullpucky of 100 means you are 100% full of bullpucky. The bullpucky scale is continuous between those two values.
Sound familiar? That's because Jeremy Kalgreen did something similar with his hilarious and beautifully laid out website, whosmorefullofshit.com. But I go two steps further than Kalgreen, who personally approved of my decision to duplicate his scoring system.
First, Kalgreen only used PolitiFact report cards. In an attempt to account for variation among fact checkers, Malark-O-Meter averages scores based on both PolitiFact and The Fact Checker, and can easily be extended to incorporate any number of fact checkers. Second, Malark-O-Meter doesn't just calculate bullpucky scores. I measure our uncertainty in bullpucky scores due to the small sizes, and in our comparisons of one individual or party to another.
I encourage you to navigate this website if you want to learn more details about my scoring and statistical methods, especially if you aren't well-versed in statistics. Then, I urge you to come back here and read my first official analysis.
Tonight, I analyze the comparative bullpucky of the 2012 presidential candidates and their running mates overall, and specific to their performance in the 2012 debates so far. This analysis is especially salient now given that there is one more debate tomorrow, and just 15 days until election day, November 6th.
Let's start by measuring the overall bullpucky of each candidate, and establishing a range of values for the bullpucky score that we can be reasonably certain they have, given the available data. Before we use fancy statistical methods to estimate probability distributions of bullpucky, let's look at the bullpucky scores we observe directly from the report cards. Below are bullpucky scores calculated from the candidates' report cards from PolitiFact and The Fact Checker, respectively.
Want to read the rest of the analysis? Go to:
http://www.malarkometer.org/...
There is also an analysis of whether challengers spew more malarkey than incumbents.
http://www.malarkometer.org/...