In this edition of ZIPs, we review the results of the experiment we conducted on Monday, which involved asking people to play a game in which they have to guess which posts got more views.
So, if you missed Monday's edition, you may want to check it out first by clicking here. Also, if you have the time, you may also enjoy playing the game that is included in Monday's edition.
Zero Impact Posts (ZIPs): May 26-8, 2012
Welcome to the twelfth edition of "Zero Impact Posts" (ZIPs). If you missed the first edition, you may want to read it in order to learn about the many positive aspects of ZIPs and thus the inspiration for the ZIP series. See the fourth edition for the raison d'etre for the ZIP series. See the tenth edition to learn why the ZIP list is not a failure list.
According to jotter, during the May 26th week and the June 2nd week, there were 2576 posts, of which 2497 received recommendations and 1476 had more than 9 commenters.
And according to me, there were only 22 posts that have had no recommendations and no comments so far to date. As many of these posts may have gotten lost in the rapids of the Recent Diary List or perhaps required a niche reader to appreciate them, many of them may have the potential for impact, even great impact. In that spirit, they are presented below.
Remember, it's never too late to recommend a post. In fact, since the series began, some posts are no longer ZIPs, as they have been recommended after having been discovered from the ZIP list.
Rollovers: To see the first few words in a post, rest your cursor momentarily over the post title. In general, wherever you see underlined text, you can rest your cursor over the text for more info. (Unfortunately, I did not have time this week to fill in the category column, which explains why they are all "#" signs.)
The Results of the View Experiment
(See the previous edition for a description of the experiment.)
Overall, we did well. There were 20 people, including myself, who participated, with the result being that the average number of correct answers was 7.2 (out of 9 possible), which is 80% correct. The median (i.e., middle value) and mode (i.e., most common value) was also 7 correct. In other words, people generally got around 7 out of 9 correct.
Only 2 participants got less than 6 correct, and there were 4 participants who got all 9 correct.
The following table shows the results. The first row is the question #s and the following rows show a "√" if the person got the right answer. (If the commenter did not specify which questions were missed, the "?" symbol is shown.)
ID |
ET |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
correct |
% |
1 |
Mon 22:56 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
9 |
100% |
2 |
Mon 23:22 |
|
√ |
√ |
√ |
√ |
√ |
√ |
√ |
|
7 |
78% |
3 |
Mon 23:33 |
√ |
√ |
√ |
√ |
|
√ |
√ |
√ |
√ |
8 |
89% |
4 |
Mon 23:48 |
√ |
√ |
√ |
√ |
|
√ |
√ |
|
√ |
7 |
78% |
5 |
Tue 00:09 |
√ |
√ |
√ |
√ |
|
|
|
√ |
√ |
6 |
67% |
6 |
Tue 00:10 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
|
√ |
8 |
89% |
7 |
Tue 01:34 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
9 |
100% |
8 |
Tue 07:39 |
|
√ |
√ |
√ |
|
|
√ |
|
√ |
8 |
89% |
9 |
Tue 07:57 |
√ |
√ |
√ |
√ |
|
|
√ |
|
√ |
6 |
67% |
10 |
Tue 08:08 |
|
√ |
√ |
|
√ |
√ |
|
√ |
|
5 |
56% |
11 |
Tue 08:21 |
|
√ |
√ |
|
|
√ |
|
√ |
|
4 |
44% |
12 |
Tue 09:41 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
|
√ |
8 |
89% |
13 |
Tue 10:05 |
√ |
√ |
√ |
√ |
|
√ |
|
√ |
|
6 |
67% |
14 |
Tue 11:58 |
√ |
|
√ |
√ |
√ |
√ |
√ |
|
√ |
7 |
78% |
15 |
Tue 12:20 |
|
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
8 |
89% |
16 |
Tue 1:19 |
? |
? |
? |
? |
? |
? |
? |
? |
? |
6 |
67% |
17 |
Tue 1:21 |
|
√ |
√ |
√ |
√ |
√ |
√ |
|
√ |
7 |
78% |
18 |
Tue 1:38 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
9 |
100% |
19 |
Tue 1:41 |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
√ |
9 |
100% |
20 |
Tue 5:21 |
|
√ |
√ |
√ |
√ |
√ |
√ |
|
√ |
7 |
78% |
|
average |
63% |
95% |
100% |
89% |
68% |
89% |
79% |
63% |
79% |
7.2 |
80% |
Hypothesis confirmed.
If you look at the bottom row (i.e., the one highlighted in grey), it shows the percentage of people who got the right answer for each question. So, for example, it shows 100% for the 3rd question, because everyone got the right answer for the 3rd question. On the other hand, only 63% of participants got the right answer for the 1st question.
Though some questions were evidently harder than others, each question was answered correctly by the majority of participants. In other words, the consensus was correct on every question. I think this helps to confirm the hypothesis that the view count is affected by post title, first words, tags, and post time.
Were the questions with recommended/rescued posts harder?
Naturally, when a post is recommended or rescued, this boosts the view count. Thus, much of the view count would not be due to the provided info (i.e., title, first words, tags, time), but rather due to being on a special list (i.e., the rec list or Community Spotlight).
There were 5 questions with posts that were recommended/rescued and 4 that did not have such posts. How did they compare in terms of how often people got the question right? The 5 questions that had recommended/rescued posts were 1, 2, 3, 4, and 8. On average, people got 82% of these questions right. For the other 4 questions, people got 79% right. So, it does not seem like the questions with recommended/rescued posts were more difficult. (Of course, this is a very small sample of data.)
Selection bias?
As you may know, some people posted their answers after I had posted the correct answers. Of course, I don't think anyone would cheat, since the stakes are so low. (Sorry, there's no prize for getting them all right.) But I was thinking, there could be selection bias, in the sense that if you get them all right, you may be more likely to want to tell the world, but if you get only half right, you may be less enthusiastic about posting the results.)
So, just to see if there's any difference, I compared the averages of the scores before and after posting the results. All rows up to and including row 13 were before I posted the results. For these rows only, the average number correct was 7.0. All rows from 14 onwards were after I posted the results. The average number correct from rows 14 to 20 was 7.6. So, the "before" average is 7.0 and the "after" average is 7.6. That's not hugely different. So, I'm thinking there was not much selection bias.
How much variation of individual scores is due to luck?
If you ask 20 people to flip 9 coins, some people are going to get more heads than others. It does not mean that the people who got the most heads are more skilled or tried harder.
So, just for the fun of it, I thought I would devise a coin-flipping experiment. Suppose that the coins are weighted, so that some are more likely to land on heads than others. Let's say they are weighted in the same way as shown in the bottom row of the results table above (i.e., 63%, 95%, 100%, 89%, 68%, 89%, 79%, 63%, and 79%). Then, suppose 20 people flip each of these coins and report how many were heads. If they did this, how many people are likely to get them all right? How many would get only half right?
So, I simulated the coin-flipping experiment using Microsoft Excel. I ran the experiment 50 times and took the average of the results. Here are the results (which I will explain in the next paragraph):
Experiment |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
view experiment |
0 |
0 |
0 |
1 |
1 |
3 |
5 |
5 |
4 |
weighted coin-toss |
0.0 |
0.0 |
0.0 |
0.2 |
0.9 |
3.6 |
5.6 |
6.3 |
2.4 |
So, in the "view experiment" held on Monday, there were 0 people who got only 1 question right, 0 people who got only 2 questions right, 0 people who got only 3 questions right, 1 person who got 4 questions right, 1 person who got 5 questions right, 3 people who got 6 questions right, 5 people who got 7 questions right, 5 people who got 8 questions right, and 4 people who got 9 questions right. (By the way, I used only the results for 19 people. The results for one person were excluded, as this person did not specify which questions they missed. See row ID 16 in the results table that was filled with "?" symbols.)
In the "weighted-coin toss" experiment that I did, there were 0.2 people who got 4 questions right, 0.9 people who got 5 questions right, 3.6 people who got 6 questions right, 5.6 people who got 7 questions right, 6.3 people who got 8 questions right, and 2.4 people who got 9 questions right. (The reason that there are fractions of people is because I took an average of 50 runs of the experiment.)
Anyway, you can see that the results are fairly similar between the "view experiment" and the "weighted-coin toss" experiment. In other words, we may be like basketball players who have the same shooting percentage. If 20 of us are asked to shoot from 9 spots on the court, then there will be some variation, just out of chance, even if we all have the same skill level.
On the other hand, I'm sure the results are also consistent with the possibility that there is variation in skill and effort. So, there's no conclusive answer to the question regarding how much of the variation in scores is due to luck verses skill and effort.
Summary
The most interesting aspect of the experiment is that it shows that we can predict which kinds of posts will get more views. Perhaps, we can improve with some practice too. So, maybe I will have similar experiments like this one in the future.
I was planning on posting the various reasons people gave for the questions they answered correctly, but I just did not have the time to do that. Yet, that would be an interesting exercise and could also be helpful.
By the way, the person at the top of the results table was me, because I was the first to play the "guess-which-post-got-more-views" game. As it turns out, I got them all right. I know people may be suspicious that I peaked at the answers. But I did not! I swear it! I collected the data using a program I wrote, in such a way that I had no idea which post had more views.
On the other hand, I may have had an advantage, simply because I had looked at the post info over several days as I was putting the ZIP edition together. Also, I spent some time thinking about each question and writing down my reasons for picking each post. Also, the next day, I reviewed my answers and I changed one of them (i.e., the answer to the first question), after some careful consideration. So, I probably spent more time than anyone.
The point is that spending some time considering the post info (i.e., post title, first words, tags, time) may help to boost the view count of your posts. You may even want to come back to your draft the next day and look at it again, to see if there's anything that can be changed to boost the view count.
Personal note: I have a plane to catch today. So, my availability will be limited, but I will try to check in from time to time. But as of this week, I'm all caught up on the ZIPs. Yea!!!
Past editions: Zero Impact Posts