httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marian Marinov>
Subject Re: URL scanning by bots
Date Fri, 03 May 2013 08:55:01 GMT
On 05/03/2013 07:24 AM, Ben Reser wrote:
> On Tue, Apr 30, 2013 at 5:23 PM, André Warnier <> wrote:
>> Alternatives :
>> 1) if you were running such a site (which I would still suppose is a
>> minority of the 600 Million websites which exist), you could easily disable
>> the feature.
>> 2) you could instead return a redirect response, to a page saying "that one
>> was sold, but look at these".
>> That may be even more friendly to search engines, and to customers.
> My point isn't that there aren't alternatives, but that 404's are
> legitimate responses that legitimate users can be expected to receive.
>   As such you'll find it nearly impossible in my opinion to convince
> people to degrade performance for them as a default.  If it isn't a
> default you're hardly any better off than you are today since it will
> not be widely deployed.
> If you want to see a case where server behavior has been tweaked in
> order to combat miscreants go take a look at SMTP.  SMTP is no longer
> simple, largely because of the various schemes people have undertaken
> to stop spam.  Despite all these schemes, spam still exists and the
> only effective counters has been:
> 1) Securing open-relays.
> 2) Removing the bot-nets that are sending the spam.
> 3) Ultimately improving the security of the vulnerable systems that
> are sending the spam.
> All the effort towards black lists, SPF, domainkeys, etc... has been
> IMHO a waste of time.  At best it has been a temporarily road block.
If Apache by default delays 404s, this may have some effect in the first month or two after
the release of this change. 
But then the the botnet writers will learn and update their software.
I do believe that these guys are monitoring mailing lists like these or at least reading the
change logs of the most 
popular web servers.
So, I believe that such change would have a very limited impact on the whole Internet or at
least will be combated 
fairly easy.


View raw message