httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier>
Subject Re: URL scanning by bots
Date Wed, 01 May 2013 14:16:27 GMT
Graham Leggett wrote:
> On 01 May 2013, at 1:51 PM, André Warnier <> wrote:
>> But *based on the actual data and patterns which I can observe on my servers (not
guesses), I think it might have an effect*.
> Of course it might have an effect - the real important question is will it have a *useful*
> A bot that gives up scanning a box that by definition isn't vulnerable to that bot (thus
the 404) doesn't achieve anything useful, the bot failed to infect the host before, it fails
to infect the host now, nothing has stopped the bot moving to the next host and trying it's
luck there. Perhaps it does achieve a reduction in traffic for you, but that is for you to
decide, and the tools already exist for you to achieve this.

Let me take this line of reasoning "ad absurdum" : the best strategy then for the bot 
would be not to scan at all, and just give up ahead of time, wouldn't it ?

Instead, isn't the logical explanation more like this :

The bot can not give up.  It's very purpose is to identify servers which have 
vulnerabilities that would allow a more targeted attempt at breaking into that server, 
right ?  In order to do that, it /must/ try a number of potentially-vulnerable URLs on 
each server, and it must wait to check how they respond.  It it walks off before waiting 
for the response, it has not achieved its main purpose, because it doesn't know the 
response to its question.

If it tries just one URL per server, and walks off if the response takes longer than some

pre-determined value, then it all depends on what this value is.
If the value is very small, then it will miss a larger proportion of the potential 
candidates. If the value is larger, then it miss less candidate servers, but it will be 
able to scan comparatively less servers within the same period of time.

> To put this into perspective, Rackspace will give me a midrange virtual server instance
with 8GB of RAM for $350-ish per month. If I wanted 10 000 of these, that's a $3.5m dollar
a month server bill. Or I could break into and steal access to 10 000 servers in my botnet,
some far larger than my 8GB ballpark, and save myself $3.5m per month. Will attempts by sites
across the net to slow down my bots convince me to stop? For $3.5m worth of computing power
that I am getting for free, I think not.

Ah, but you are disregarding two important factors here:
1) spending 3.5 M$ to rent 10,000 servers is legal, and will not lead you to jail.
If anything, it will probably earn you some nice discount coupons.
In contrast, deploying and running a botnet of 10,000 servers is a criminal activity, and

can result in a big fine and being put in jail.
If am going to take a certain risk of having to pay millions of $ in fines and damages, 
and spend some time in jail to boot, I would want to have a corresponding probability of 
making a profit. Not you ?
2) you seem to believe that deploying a botnet of 1000 bots costs nothing.  Who is going 
to write the code for your bot ? or alternatively, how much money would you be wanting to

spend in order to buy the code ? (You can find prices in Google)
And would you know exactly who are the people you would be buying that code from ?

Let me pick on another element of your message : "the tools already exist for you to 
achieve this"
Yes, they do.  There are plenty of tools available, which achieve a much better protection

for a server than my proposal ever would (although that is not really my purpose).

But have you already looked at these tools, really ?
Most of these tools require at least a significant expertise (and time) on the part of the

webserver administrator to set them up correctly.  Many of the most effective ones also 
consume a significant amount of resources when running. Some of them even cost money.

Which in the end and practically leads to the current real-world situation : there are 
hundreds of millions of webservers on the Internet which do /not/ implement any of these 
tools.  Which is one of the elements which makes running these URL-scanning bots be a 
profitable proposition, until now.

In contrast, my proposal would not require any expertise or any time or any money on the 
part of whoever installs an Apache server.  They would just install the default server "as

is", as they get it from the Apache website or from their preferred platform distribution.
And it would slow down the bots (until someone proves the opposite to me, I'll stick with

that assertion).

View raw message