tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Benussi" <mark_benu...@hotmail.com>
Subject Re: robots.txt
Date Thu, 14 Jul 2005 18:22:23 GMT
With follow = false or something cant remember the exact syntax

----Original Message Follows----
From: Tim Funk <funkman@joedog.org>
Reply-To: "Tomcat Users List" <tomcat-user@jakarta.apache.org>
To: Tomcat Users List <tomcat-user@jakarta.apache.org>
Subject: Re: robots.txt
Date: Thu, 14 Jul 2005 14:14:32 -0400

Place a robots meta tag in the head tag of your html.

-Tim

Eric J. Pastoor wrote:
>The msn bot has been hammering one of my webapps for about 2 weeks now. I 
>have a calendar style transaction page that is not password protected. 
>Essentially this calendar goes on forever. This means that the msn bot is 
>basically going on forever.  It is making my log files insanely large.  Is 
>there a way to create a robots.txt file for basically one servlet instead 
>of everything?

---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Mime
View raw message