nutch-agent mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From WebExpertsAmerica <exp...@WebExpertsAmerica.com>
Subject Your Nutch Crawler is Out of Control - Apache Notified
Date Fri, 23 Sep 2005 19:25:49 GMT

You crawler is ignoring our robots.txt file.

http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)"
128.95.1.189

You are eating bandwidth at our domain in incredible amounts. This is
rude. 

Please stop or we will be forced to block your IP and the crawler you
are using.

Best Regards,

Web Experts America

>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<
WebExpertsAmerica.com
Whole Lot More for a Whole Lot Lessİ
$6/hr Professional Web Services
http://www.WebExpertsAmerica.com

Testimonials:
http://www.WebExpertsAmerica.com/testimonials.htm

Website Solutions:
http://www.WebExpertsAmerica.com/services.htm

Chat:
WebExpertsNOW
AOL, MSN (Hotmail), and Yahoo
*Contact us anytime via chat. However, we DENY, BLOCK, and BAN anyone
that adds us to their Friend/Buddy list. Nothing personal, a security
policy to protect our chat connectivity from competitor abuse.

Terms of Service:
http://www.WebExpertsAmerica.com/tos.htm

Confidential:
The information contained in this message is privileged and confidential
and protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify us
immediately by replying to this message and then delete it from your
computer.



Mime
View raw message