tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leon Rosenberg" <rosenberg.l...@googlemail.com>
Subject Re: Web spiders - disabling jsessionid
Date Fri, 01 Dec 2006 14:12:26 GMT
you believe everything you've been told ?:-)

google has 3 (at least 3 known) user agents : google, mozzila with
google-bot in the agent string (the one you sent) and another one,
which is just Mozilla/5.0.

google uses this 3rd agent to check your site from another ip adress,
whether you do some ugly seo stuff, like cloacking etc.

If it detects that you deliver different content to his
mozilla-disguised bot, your chances to be thrown out of the index are
pretty high.

so please don't do it, if you rely on being found.

regards
Leon

On 12/1/06, Tim Funk <funkman@joedog.org> wrote:
> Wrong. Google is very clear about not hiding user agent - as well as a
> the other major bots.
>
> Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html
>
> Just just for Googlebot in the user-agent header.
>
> -Tim
>
> Leon Rosenberg wrote:
> > On 12/1/06, Tim Funk <funkman@joedog.org> wrote:
> >> The easiest is the filter and custom HttpServletResponse which overrides
> >> encodeURL() to do nothing.
> >>
> >> It could be made one step smarter by checking if the User agent is a
> >> search engine bot to selectively execute or not.
> >>
> >
> > How do you want to achieve that?
> > They disguise themself as normal browsers....
> > Leon
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message