Website Traffic – Are Competitive Intelligence Tools Accurate? (Survey)

Sean Weigold Ferguson — Wed, 16 Sep 2009 06:01:32 +0000

For the last several months, I have been conducting research to find out how accurately online competitive intelligence tools can predict website traffic. I am at the point where I am ready to begin reporting some preliminary results. Before I do, I would like to gain a better understanding how these tools are perceived by those familiar with them.

Based on my review of the literature, the most often cited tools for estimating website traffic are:

If you are familiar with at least one of these tools, I would love to have your input. The survey below is very short; essentially one question. If you donate just one minute of your time to fill it out, I will greatly appreciate it and promise that you will receive a copy of the results.

I would like to collect a couple of hundred responses in order to report statistically significant results. If you know someone who might be interested in the topic, please point them in this direction. The easiest way to help is by simply retweeting this post, or sharing it your favorite social network.

Take the Survey

Website Traffic – Are Competitive Intelligence Tools Accurate? (Survey)

was brought to you by WellonTop Internet Research

Competitive Intelligence Tools – Alexa, Compete, Google Trends & Quantcast

Sean Weigold Ferguson — Mon, 06 Jul 2009 01:08:05 +0000

The veracity of the data provided by free competitive intelligence tools is often called into question. In some cases it seems like traffic estimates are so out of line with reality that the entire industry is just bunk. What, if any value do they actually offer us?

StumbleUpon v. Digg – A Case Study

The purpose of this case study is to describe the predictions made by five free competitive analysis tools. We will do this by comparing estimates of unique visitors to two popular social bookmarking websites. At the end I’ll share a small but valuable secret that will allow us to test any hypotheses we form.

Quantcast

A cursory glance at Quantcast’s visitor estimates for StumbleUpon.com isn’t particularly interesting. According to Quantcast, StumbleUpon receives less than a million unique US visitors/month. Even worse, the website has shown a dramatic decline in traffic since March of this year. For a website with nearly 8 million members worldwide, StumbleUpon’s numbers are at best disappointing, especially compared to Digg’s 12.5 million unique US visitors/month.

Compete.com

Comparing StumbleUpon and Digg on Compete.com tells a slightly different story. In strong contrast to Quantcast’s directly measured data, Compete estimates Digg receives almost 39 million unique US visitors/month. Compete’s estimate is more than 3x the actual traffic volume. Compete also comparitively exaggerates StumbleUpon’s traffic at 5 million visitors; more than 6x Quantcast’s estimate.

Alexa

Adding another tool only makes things more confusing. Alexa estimates the global reach of Digg and StumbleUpon to be .53% and .19% respectively. It’s important to note that we cannot directly compare Alexa’s “daily reach” (even averaged over 30 days) with “unique monthly visitors.” The problem is that each measure two distinct (however related) variables. In order to properly compare the two, we need to account for how many visits unique visitors are responsible for. If that doesn’t make sense yet, don’t worry, I still have a hard time wrapping my head around it too. Leave a comment and I’ll make sure to follow up with a post explaining it.

For the purposes of this limited case study we’ll assume there are approximately 1.6 billion internet users worldwide, half of whom (800 million) are online on any given day. Digg should be receiving 4.2 million unique global visitors/day compared to StumbleUpon’s 1.5 million. We know that Alexa’s estimate for Digg is more than double the true number, but what about StumbleUpon?

Google Trends

At this point, it seems like we may as well just throw our hands up in the air and acknowledge that free competitive analysis estimates are largely worthless. Fortunately, at WoT, the words “giving” and “up” don’t hang out together. They’re much more fun by themselves anyway. Luckily for us we stumbled upon (punny) something spectacular in the process.

As usual, data-superhero Google comes flying to our rescue. Comparing Digg and StumbleUpon on Google Trends begins to clear up this mess. At the end of March, StumbleUpon has less than half the unique US visitors/month of Digg. The gap closes quite suddenly in mid-April in part due via a combination of three major events:

April 13^th – StumbleUpon is bought back from eBay by its founders
April 22^nd – Version 3.29 of the StumbleUpon Firefox toolbar is released
April 27^th – Web Stumbling (without the toolbar) receives major enhancements

Competitive Analysis Tools Compared

Not all of our competitive intelligence tools have data past May. To compensate we will only use the month of April, identified by the period immediately prior to StumbleUpon’s traffic spike.

Immediately we see the degree to which Compete has egregiously overestimated Digg’s traffic. Google Trends is almost certainly an underestimation. What about the data on global visitors?

Alexa and Google Trends both drastically underestimate Digg’s global traffic. While we don’t have any way to know for sure, I would posit that they also underestimate StumbleUpon’s data.

Interpreting Results

So what exactly can we take away from this case study? For one, we can be pretty confident that StumbleUpon received significantly fewer unique global visitors in the first few weeks of April than Digg. We can even take a stab at the difference. Assuming that StumbleUpon data is skewed in proportion to Digg data biases, our corrected global estimate for Google Trends is 8.4 million users. Applying the same methodology to Alexa gives a global visitor estimate of 7.3 million.

We know that about 50% of Digg’s unique users are from outside the US. Google Trends’ estimates can account for 84% of global traffic but only 53% of US traffic. To me this indicates a slight measurement bias in favor of international users. If StumbleUpon’s Google Trends data is similarly biased, our corrected US traffic estimate is 4.4 million users. When we apply the same technique Compete, the estimate falls from 5.9 million to 3.9 million users. Quantcast’s corrected estimate seems somewhat low at 2.2 million users.

Forming Hypotheses

It’s fairly surprising just how consistent our corrected data is. Based on the information gathered, I’m going to hypothesize that during the measured time period:

StumbleUpon received between 7 to 9 million unique global visitors
StumbleUpon received between 2 to 5 million unique US visitors
Free competitive analysis tools can be useful if biases can be compensated for

A Special Bonus

In this case, I can test the first two hypotheses. I found a very valuable little snippet at the bottom of StumbleUpon’s source code. I’ll give you a hint: It begins with