httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eike Frost>
Subject Re: [users@httpd] How to disable web accelerators
Date Fri, 28 Jul 2006 16:33:23 GMT

On Friday 28 July 2006 00:24, Gerard Seibert wrote:
> I am not sure if this is the right place to ask or not. How do I go
> about disabling the use of web accelerators like FasterFox or the Google
> Web Accelerator? I have noticed that several sites now have a warning pop
> up if you are using such a device and refuse you access until it is
> turned off.

In addition to the User-Agent blocking already proposed, some sites also use a 
trick at a higher layer. Namely, they exploit that web accelerators usually 
follow links indiscriminantly, even ones that are not visible or clickable (a 
transparent 1-pixel PNG or a CSS-disabled link, for instance). If you add 
such a link to your webpage it is safe to assume that only robots "click" it; 
and once the linked URL has been requested you can do further processing 
(such as displaying a notice to the user on the next dynamic page load of the 
main page, or adding the user to a temporary blocklist; how that is 
implemented is left up to you -- you could check cookies, redirect with 
additional GET-parameters set, or have more intelligent handling inside your 
dynamic pages, for example).
If you also make sure that the robots.txt excludes at least that one file, 
well-behaving bots will never trigger the script behavior. Combined with 
User-Agent checking, this will catch most of the cases.


View raw message