The veracity of the data provided by free competitive intelligence tools is often called into question. In some cases it seems like traffic estimates are so out of line with reality that the entire industry is just bunk. What, if any value do they actually offer us?
StumbleUpon v. Digg – A Case Study
The purpose of this case study is to describe the predictions made by five free competitive analysis tools. We will do this by comparing estimates of unique visitors to two popular social bookmarking websites. At the end I’ll share a small but valuable secret that will allow us to test any hypotheses we form.
Quantcast
A cursory glance at Quantcast’s visitor estimates for StumbleUpon.com isn’t particularly interesting. According to Quantcast, StumbleUpon receives less than a million unique US visitors/month. Even worse, the website has shown a dramatic decline in traffic since March of this year. For a website with nearly 8 million members worldwide, StumbleUpon’s numbers are at best disappointing, especially compared to Digg’s 12.5 million unique US visitors/month.
Compete.com
Comparing StumbleUpon and Digg on Compete.com tells a slightly different story. In strong contrast to Quantcast’s directly measured data, Compete estimates Digg receives almost 39 million unique US visitors/month. Compete’s estimate is more than 3x the actual traffic volume. Compete also comparitively exaggerates StumbleUpon’s traffic at 5 million visitors; more than 6x Quantcast’s estimate.
Alexa
Adding another tool only makes things more confusing. Alexa estimates the global reach of Digg and StumbleUpon to be .53% and .19% respectively. It’s important to note that we cannot directly compare Alexa’s “daily reach” (even averaged over 30 days) with “unique monthly visitors.” The problem is that each measure two distinct (however related) variables. In order to properly compare the two, we need to account for how many visits unique visitors are responsible for. If that doesn’t make sense yet, don’t worry, I still have a hard time wrapping my head around it too. Leave a comment and I’ll make sure to follow up with a post explaining it.
For the purposes of this limited case study we’ll assume there are approximately 1.6 billion internet users worldwide, half of whom (800 million) are online on any given day. Digg should be receiving 4.2 million unique global visitors/day compared to StumbleUpon’s 1.5 million. We know that Alexa’s estimate for Digg is more than double the true number, but what about StumbleUpon?
Google Trends
At this point, it seems like we may as well just throw our hands up in the air and acknowledge that free competitive analysis estimates are largely worthless. Fortunately, at WoT, the words “giving” and “up” don’t hang out together. They’re much more fun by themselves anyway. Luckily for us we stumbled upon (punny) something spectacular in the process.
As usual, data-superhero Google comes flying to our rescue. Comparing Digg and StumbleUpon on Google Trends begins to clear up this mess. At the end of March, StumbleUpon has less than half the unique US visitors/month of Digg. The gap closes quite suddenly in mid-April in part due via a combination of three major events:
- April 13th – StumbleUpon is bought back from eBay by its founders
- April 22nd – Version 3.29 of the StumbleUpon Firefox toolbar is released
- April 27th – Web Stumbling (without the toolbar) receives major enhancements
Competitive Analysis Tools Compared
Not all of our competitive intelligence tools have data past May. To compensate we will only use the month of April, identified by the period immediately prior to StumbleUpon’s traffic spike.
Immediately we see the degree to which Compete has egregiously overestimated Digg’s traffic. Google Trends is almost certainly an underestimation. What about the data on global visitors?
Alexa and Google Trends both drastically underestimate Digg’s global traffic. While we don’t have any way to know for sure, I would posit that they also underestimate StumbleUpon’s data.
Interpreting Results
So what exactly can we take away from this case study? For one, we can be pretty confident that StumbleUpon received significantly fewer unique global visitors in the first few weeks of April than Digg. We can even take a stab at the difference. Assuming that StumbleUpon data is skewed in proportion to Digg data biases, our corrected global estimate for Google Trends is 8.4 million users. Applying the same methodology to Alexa gives a global visitor estimate of 7.3 million.
We know that about 50% of Digg’s unique users are from outside the US. Google Trends’ estimates can account for 84% of global traffic but only 53% of US traffic. To me this indicates a slight measurement bias in favor of international users. If StumbleUpon’s Google Trends data is similarly biased, our corrected US traffic estimate is 4.4 million users. When we apply the same technique Compete, the estimate falls from 5.9 million to 3.9 million users. Quantcast’s corrected estimate seems somewhat low at 2.2 million users.
Forming Hypotheses
It’s fairly surprising just how consistent our corrected data is. Based on the information gathered, I’m going to hypothesize that during the measured time period:
- StumbleUpon received between 7 to 9 million unique global visitors
- StumbleUpon received between 2 to 5 million unique US visitors
- Free competitive analysis tools can be useful if biases can be compensated for
A Special Bonus
In this case, I can test the first two hypotheses. I found a very valuable little snippet at the bottom of StumbleUpon’s source code. I’ll give you a hint: It begins with <!–Start Quantcast tag–>
Secret StumbleUpon Data on Quantcast!
Notes:
Some charts on this page have been modified. Links to the original sources have been included.
Photo: kevinzhengli on flickr.com
Inside Facebook
