The Question
I embarked upon my study of online competitive intelligence tools in late June of 2009. My muse for this project was a study by SEOmoz comparing the website analytics of 25 Internet marketing blogs with estimates made by intelligence tools such as Compete Site Profile and Alex Site Information. SEOmoz’s conclusion was that:
Based on the evidence we’ve gathered here, it’s safe to say that no external metric, traffic prediction service or ranking system available on the web today provides any accuracy when compared with real numbers.
I found the study exciting, but the results surprising and somewhat counterintuitive. Because research thrills me, I began pondering ways I could replicate their results with a larger and more diverse sample. I immediately saw an opportunity in a relative newcomer to the competitive intelligence industry: Quantcast.
Competitive Intelligence
Companies such as Quantcast that offer web analytics, competitive intelligence, and/or market research services generally gather information from one or more of sources:
- Panels of Internet users
- Aggregate ISP data
- On-site direct measurement
Each of these methods has strengths and weaknesses. The current trend in the world of competitive intelligence is to eschew relying on a single data source. Instead, companies are choosing to integrate two or more. Hypothetically, this allows them to compensate for the weaknesses of a single method.
Quantcast
Quantcast could be considered the pioneer of direct measurement in the competitive intelligence industry. Quantcast even compensates for the major weakness of direct measurement by employing cookie corrected audience data, taking into account …numerous factors including the frequency of visitation and the respective balance between work and home access to build a translation of cookies to people that is unique to each digital media property.
The Study
I began this study with the assumption that directly measured traffic data from Quantified websites are 100% accurate. I now know that reaching 100% accuracy with web analytics is virtually impossible. However, it is likely that a metric such as page views, which is not dependent on tracking cookies or JavaScript is as close to canon as is possible under current technological constraints.
Under this assumption, I have compared Quantcast’s direct measurements of websites to monthly traffic estimates given by services such as Alexa, Compete, and Google Ad Planner. Since June, I have conducted several pretests using smaller samples, culminating in the current analysis of more than 1,350 root domains. My correlations have remained fairly consistent regardless of sample size and month of data collection.
Further Exploration
I have gathered data on my sample from many other sources including search engines, social media services, and miscellaneous third-party tools. These additional variables give insight into the factors that mediate intelligence tool estimates. They also present the opportunity to conduct future analyses by identifying the factors that correlate with website traffic and user engagement.
I will be posting the results of my research this month, mediated only by the pace at which I can write about and display them. I strongly encourage you to leave questions and requests as comments on this post. I will make sure to address them in my analyses.
Special Thanks
I would like to thank Aaron Prebluda of Compete.com, and Danny Dover of SEOmoz for their support and patience. I truly appreciate your aid and advocacy. I am confident that you will find it was well worth your time.
How You Can Help
Interested in contributing to this study? I would greatly appreciate your taking my survey on the topic. It should take you less than a minute to complete. Make sure to send this post to your friends too. Thanks!
Inside Facebook
