My friend Philipp Lenssen wrotes some days ago about the German magazine DotNetPro which offers different html output when Google's crawler is fetching the site.

“Neue Mediengesellschaft mbH” is a German publisher of tech print magazines like Com!, Internet World Business, MacUp and DotNetPro. Fair enough, except for the fact their DotNetPro.de website (a PageRank 6 site, with about 25,400 pages indexed in Google) happens to use a form of hidden keywords and cloaking the Google webmaster guidelines consider abuse. My friend Markus repeatedly stumbled upon their website searching Google.de for things like asp.net output caching kontrollieren. While DotNetPro.de appears on the first Google results page (not on all datacenters I hit, but on some of them), the word “output”, for example, does not appear anywhere within the page or its HTML source when you click on it. Instead, DotNetPro urges you to subscribe to their print magazine to then be able to download a PDF of this article. [...]

I still don't understand why this will not be fixed because the issue has already seen last year. There are still a lot of companies trying to move in top of the search result list, see an older post at Google Blogoscoped.

If you want to see how a web site will be seen from the Google crawler simple have a look at the cached version of a web site or use Fiddler and add following custom rule:

static function OnBeforeRequest(oSession:Fiddler.Session)
{
    oSession.oRequest["User-Agent"] = "Googlebot";
}

No Comments