How accurate is Alexa for traffic measurement?

Ashley,

As per our offline discussions, Alexa can be useful if used to analyse trends; but also dangerous if used as an absolute means of comparing sites.

At www.Top-Consultant.com we use Webtrends to track our site stats and I’m quite happy to share our average weekday traffic which ranges from 6,500 to 10,000 unique visitors (higher on newsletter days). So in principle our ranking should be quite a lot higher than e-consultancy’s; but in reality we have a lower ranking (which should have alarm bells ringing!).

The fact that e-consultancy is read by people at the cutting edge of e-business would imply that you are more likely to have readers who use tools like Alexa; and you’re also more likely to have a strong US readership. These are the types of factors that can explain such apparently perverse rankings.

The implications of this are straightforward... as soon as there are reasons to think that the demographics of two sites differ significantly, then a tool like Alexa cannot be used to square up one site against the other. So Workthing vs Totaljobs works quite well, but only because the reader demographics are VERY similar. E-Consultancy’s ranking is not that far off Workthing’s (last time I checked), yet your stats would confirm there is a world of difference in the actual visitor levels the two sites receive. Again this is down to the difference in your readership demographics

* So when is Alexa useful? *

Alexa can be useful for monitoring site traffic trends for one particular site - and for generating a list of similar sites by seeing which other sites Alexa users read once they’ve read the site you are checking up on. The average page views data is also much less likely to be skewed by the Alexa readership bias, so can be used to compare different sites. Above all, though, I would say Alexa is useful for checking on any site you are thinking of partnering with / advertising on. If a site does not appear in the top 100,000 sites then for us that sets alarm bells ringing and we would not proceed any further with that site without seeing independently produced / audited site stats.

Does this tie-in with other readers’ experiences?

On 11:34:12 14 June 2004 Ashley wrote:

Does anyone else use Alexa.com for (crude) comparative traffic analysis? With a little joint effort could we figure out just how accuracte / inaccurate its traffic data is?

If you haven’t seen or used Alexa before, have a look at E-consultancy’s site traffic details over the last year. Clearly as the graph looks positive I’m happy to give this link ;). You can also compare your site’s traffic details with that of other (competitor) sites. 

On the face of it this would seem to be a free service to help you benchmark your own traffic and, more interestingly, that of other sites e.g. competitors, potential partners etc. A free version of Hitwise - though of course Hitwise does a lot more.

Certainly I have heard lots of people, even owners of large sites with plenty of budget for more sophisticated media planning tools,  say they use Alexa even if just for an intial indicative probe into another site.

But how accurate are these rankings? Alexa has a page which explains how its rankings are calculated.  This includes the following disclaimers:

• Our users are disproportionately likely to visit alexa.com, amazon.com and archive.org, and traffic to these sites may be substantially overcounted.
• The Alexa Toolbar works only with the Internet Explorer browser. Sites frequented mainly by users of other browsers will be undercounted. For example, the AOL/Netscape browser is not supported, which means that Alexa collects little data from AOL users, and our traffic to aol.com is likely lower than it would be for a more representative sample.
• The Alexa Toolbar works only on Windows operating systems. Although a large majority of the Internet population currently use Windows, traffic to any sites which are disproportionately visited by users of other operating systems will be undercounted.
• The rate of adoption of Alexa software in different parts of the world may vary widely due to advertising locality, language, and other geographic and cultural factors. For example, to some extent the prominence of Korean sites among our top-ranked sites reflects known high rates of general Internet usage in South Korea, but there may also be a dispropotionate number of Korean Alexa users.
• In some cases traffic data may also be adversely affected by our "site" definitions. With tens of millions of hosts on the Internet, our automated procedures for determining which hosts are serving the "same" content may be incorrect and/or out-of-date. Similarly, the determinations of domains and home pages may not always be accurate. When these determinations change (as they do periodically), there may be sudden artificial changes in the Alexa traffic rankings for some sites as a consequence.
• The Alexa Toolbar turns itself off on secure pages (https:). Sites with more secure page views will be under-represented in the Alexa traffic data.

So my take is that for a UK site the rankings will be overly US skewed and that the rankings will also favour sites whose users include early-adopter / internet-y types. Sites whose target users are ’mass market’, and who are less likely to install the Alexa toolbar, will fare less well.

All of which means that E-consultancy is likely to be ranked much higher than it is due.

However, can you not compare like sites with like with some degree of accuracy? So sites which have the same, or very similar, target market? The sample biases might at least then be representative?

I’ve talked to people who’ve used Alexa in conjunction with other tools and, on the whole, they’ve been surprised with the apparent degree of accuracy - at least relatively speaking - that they get with Alexa.

Out of interest, I compared totaljobs.com with workthing.com on Alexa with totaljobs.com coming out a fair bit higher. Then I had a look at published audit data for the same periods on ABCe:

• ABCe audit for totaljobs.com 01/10/03 - 31/10/03
• ABCe audit for workthing.com 01/10/03 - 31/10/03

Looks to me like a fairly accurate correlation?

According to our web traffic analysis we average around 3,500 unqiue users a day generating around 10,000 page views and our Alexa ranking is around 17,000. Anyone else happy to share their stats (particularly sites likely to have a similar user skew) to see what the correlation is?

Does anyone else have views on Alexa’s accuracy or lack of? Do people use Alexa as a (crude) benchmarking tool?

Ashley