Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Date: Tue, 6 Dec 2005 22:05:42 +0000
From: Colm MacCarthaigh <colm@stdlib.net>
To: dev@httpd.apache.org
Subject: Re: using proxy/cache for apache mirrors
Message-ID: <20051206220542.GA8467@dochas.stdlib.net>
Reply-To: colm@stdlib.net
References: <4395FFA1.4040209@slive.ca>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-15
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4395FFA1.4040209@slive.ca>
User-Agent: Mutt/1.5.9i

On Tue, Dec 06, 2005 at 04:16:17PM -0500, Joshua Slive wrote:
> This is really an infrastructure topic, not an httpd-dev one, but I'd 
> like the caching experts to look over this to make sure this simple 
> configuration looks reasonable.  

I think this is a terrible, terrible, terrible idea in general. I've
always hated this recommendation. I've kept quite, but no longer ! ;-)

"we highly recommend that you mirror using a cached reverse proxy in
place of rsync" makes me wince with pain.

Problems:

	* It's vastly more complicated than neccessary and adds a burden
	  to what admins have to manage. Why should they have to worry
	  about managing a cache? They're busy enough trying to give us
	  free resources in the first place.

	* It adds massive dependencies to what a mirror server needs to run.
	  Adding modules, especially proxy, is not resource-free. These
	  things eat memory, research time and security work.

	* It defeats a huge part of the point of having a mirroring
          system in the first place. Mirroring isn't just a way of
	  decreasing bandwidth usage on the primary, it's also a means
	  of building content resilience. When www.apache.org goes down, users 
          want their mirror to work. And worst of all, in the case of
	  infrequently used mirrors, this is exactly when they'll
	  suddenly get a lot of queries - all of which will end up in IOWAIT
	  land, with a boat-load of back-end TCP connections, and no 
	  content served. That really sucks, for both them and their
	  users.

	* mod_cache + mod_proxy is trivially vulnerable to all of the latest
	  DNS cache-poisoning trickery, with no easy fix. At the very
	  least we should recommend that admins hard-code www.apache.org
	  in their /etc/hosts file, and that INFRA get some PI-space and
	  guarantee availability at a particular IP address for
	  eternity. Or deploy DNSSEC, and insist that mirrors verify the
	  records.

	* We havn't fixed all of the thundering herd problems :/

	* It's HTTP only. A lot of users use rsync and FTP to fetch
	  content from a local mirror.

	* Next time www.apache.org gets compromised, the exposure
	  will be two to four times as great compared to the rsync
	  mirrors. CacheMaxExpire can fix this problem though.

Personally, specially for the reasons of potential cache poisoning, I'd
strong advise against using this kind of technique and stay with KISS.

-- 
Colm MacC�rthaigh                        Public Key: colm+pgp@stdlib.net