[ Sponsored Links ]

Advertise here »

Re: Allowing search engines to spider 'hidden' content - best practice?

reply to message » post a new message » e-mail to a friend »
Subject:  Re: Allowing search engines to spider 'hidden' content - best practice? Ashley Friedlein - Favourite
Author:   Ashley :   view profile | all posts by this author | add to favourites
Date:   14:13:39 8 August 2007
 
On the "cloaking" thing there seems to be a small, but important, difference to serving different 'content' versus serving different 'access'. It is wrong to serve different content to Google vs. users but apparently not wrong to serve different levels of content access?

As per Adam's link below to Google News' guidelines on allowing access to subscription content, they recommend:

"The easiest way to do this (allow Google to spider restricted content but not allow users to do so) is to configure your webservers to not serve the registration page to our crawlers (when the User-Agent is "Googlebot")"

To the uneducated this sounds to me very like 'serving something different to Googlebot than to users'? (i.e. cloaking).

But is this only true for Google News and not the main index? Teddie's mainstream examples further down this thread seem to make it clear that this also applies to the main index. In fact, Teddie's answer seems to me to sum up the best way to do things currently.

Though we're now scratching our heads about the best combination of deterrents (reffering strings, user agents, cookies, sessions etc.) to actually uniquely identify a user and try and prevent as much content theft as possible whilst not over-complicating things...

Ashley
 
« Back to Internet Marketing Forum Previous Thread »
« Back to Internet Marketing Forum Post a new message »
Subscribe for only €299