httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Hartill <>
Subject NetJet
Date Thu, 12 Dec 1996 21:03:53 GMT

Isn't *anyone* else noticing NetJet on their servers ?

I've got one ISP user making 5,000 requests per hour with it; non-stop.
It's been running for 24+ hours so there's nobody at the other end! I've
been sending it 800k strings in response and can't get it to fall over :-(
(sending all that data does slow it down to a few hundred requests per
minute though). Now waiting for the ISP people to respond.

btw, anyone running a proxy server should seriously consider using the
"deny from user-agents" option to stop maniac agents from trashing other
people's sites via the proxy.

AOL are contracting out to a firm which scans sites for objectionable material.
Trouble is it wants to check the *entire* site ! and does so at high speed.
The software (CyberSpyder), from is hosed because it can't
react to HTTP/1.1 and it can't react to errors. The owner has been "trying"
to fix it for two weeks. I told him he'd be charged per request and then
the requests stopped. I'm on his "avoid this site at all costs" list now,
however last night his robot wandered into a forwarding service in Australia
and started to rapid-fire from there.

Thinking of upgrading your hardware ? ... check for nuisance robots first.

Rob Hartill.       Internet Movie Database Ltd.  

View raw message