How accurate is Alexa for traffic measurement?

Does anyone else use Alexa.com for (crude) comparative traffic analysis? With a little joint effort could we figure out just how accuracte / inaccurate its traffic data is?

If you haven’t seen or used Alexa before, have a look at E-consultancy’s site traffic details over the last year. Clearly as the graph looks positive I’m happy to give this link ;). You can also compare your site’s traffic details with that of other (competitor) sites. 

On the face of it this would seem to be a free service to help you benchmark your own traffic and, more interestingly, that of other sites e.g. competitors, potential partners etc. A free version of Hitwise - though of course Hitwise does a lot more.

Certainly I have heard lots of people, even owners of large sites with plenty of budget for more sophisticated media planning tools,  say they use Alexa even if just for an intial indicative probe into another site.

But how accurate are these rankings? Alexa has a page which explains how its rankings are calculated.  This includes the following disclaimers:

• Our users are disproportionately likely to visit alexa.com, amazon.com and archive.org, and traffic to these sites may be substantially overcounted.
• The Alexa Toolbar works only with the Internet Explorer browser. Sites frequented mainly by users of other browsers will be undercounted. For example, the AOL/Netscape browser is not supported, which means that Alexa collects little data from AOL users, and our traffic to aol.com is likely lower than it would be for a more representative sample.
• The Alexa Toolbar works only on Windows operating systems. Although a large majority of the Internet population currently use Windows, traffic to any sites which are disproportionately visited by users of other operating systems will be undercounted.
• The rate of adoption of Alexa software in different parts of the world may vary widely due to advertising locality, language, and other geographic and cultural factors. For example, to some extent the prominence of Korean sites among our top-ranked sites reflects known high rates of general Internet usage in South Korea, but there may also be a dispropotionate number of Korean Alexa users.
• In some cases traffic data may also be adversely affected by our "site" definitions. With tens of millions of hosts on the Internet, our automated procedures for determining which hosts are serving the "same" content may be incorrect and/or out-of-date. Similarly, the determinations of domains and home pages may not always be accurate. When these determinations change (as they do periodically), there may be sudden artificial changes in the Alexa traffic rankings for some sites as a consequence.
• The Alexa Toolbar turns itself off on secure pages (https:). Sites with more secure page views will be under-represented in the Alexa traffic data.

So my take is that for a UK site the rankings will be overly US skewed and that the rankings will also favour sites whose users include early-adopter / internet-y types. Sites whose target users are ’mass market’, and who are less likely to install the Alexa toolbar, will fare less well.

All of which means that E-consultancy is likely to be ranked much higher than it is due.

However, can you not compare like sites with like with some degree of accuracy? So sites which have the same, or very similar, target market? The sample biases might at least then be representative?

I’ve talked to people who’ve used Alexa in conjunction with other tools and, on the whole, they’ve been surprised with the apparent degree of accuracy - at least relatively speaking - that they get with Alexa.

Out of interest, I compared totaljobs.com with workthing.com on Alexa with totaljobs.com coming out a fair bit higher. Then I had a look at published audit data for the same periods on ABCe:

• ABCe audit for totaljobs.com 01/10/03 - 31/10/03
• ABCe audit for workthing.com 01/10/03 - 31/10/03

Looks to me like a fairly accurate correlation?

According to our web traffic analysis we average around 3,500 unqiue users a day generating around 10,000 page views and our Alexa ranking is around 17,000. Anyone else happy to share their stats (particularly sites likely to have a similar user skew) to see what the correlation is?

Does anyone else have views on Alexa’s accuracy or lack of? Do people use Alexa as a (crude) benchmarking tool?

Ashley