httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph S D Yao <>
Subject Re: [users@httpd] Stoping bots from sucking files
Date Fri, 24 Oct 2008 19:54:27 GMT
On Fri, Oct 24, 2008 at 08:29:43PM +0100, Mário Gamito wrote:
> Hi,
> I have this site that has a directory with some files.
> A few weeks ago, two web bots started sucking those files at an impressive rate.

Use a robots.txt file in your home directory.

If they ignore it, then use iptables to block them.  That takes the
strain off httpd.

** Joe Yao - Joseph S. D. Yao

The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:> for more info.
To unsubscribe, e-mail:
   "   from the digest:
For additional commands, e-mail:

View raw message