Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: pass (asf.osuosl.org: domain of minfrin@sharp.fm designates
 64.49.220.200 as permitted sender)
Message-ID: <38271.196.8.104.37.1146564091.squirrel@www.sharp.fm>
In-Reply-To: <Pine.GSO.4.64.0605021047160.8006@kleopatra.acc.umu.se>
References: <4450CF63.2060305@turner.com>
    <445136CC.6030305@sharp.fm><44520B45.6070507@turner.com>
    <4455203F.9000509@sharp.fm>
    <44560316.3030705@turner.com><445664B1.5030905@sharp.fm>
    <445665B2.3010604@turner.com><20060501171853.8fd9164a.davi@haxent.com.br>
    <Pine.GSO.4.64.0605021047160.8006@kleopatra.acc.umu.se>
Date: Tue, 2 May 2006 12:01:31 +0200 (SAST)
Subject: Re: Possible new cache architecture
From: "Graham Leggett" <minfrin@sharp.fm>
To: dev@httpd.apache.org
Cc: dev@httpd.apache.org
User-Agent: SquirrelMail/1.4.3a-12.EL4
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Importance: Normal

On Tue, May 2, 2006 11:22 am, Niklas Edmundsson said:

> I've been hacking on mod_disk_cache to make it:
> * Only store one set of data when one uncached item is accessed
>    simultaneously (currently all requests cache the file and the last
>    finished cache process is "wins").
> * Don't wait until the whole item is cached, reply while caching
>    (currently it stalls).
> * Don't block the requesting thread when requestng a large uncached
>    item, cache in the background and reply while caching (currently it
>    stalls).

This is great, in doing this you've been solving a proxy bug that was
first reported in 1998 :).

The only things to be careful of is for Cache-Control: no-cache and
friends to be handled gracefully (the partially cached file should be
marked as "delete-me" so that the current request creates a new cache file
/ no cache file. Existing running downloads should be unaffected by
this.), and for backend failures (either a timeout or a premature socket
close) to cause the cache entry to be invalidated and deleted.

> * More or less atomic operations, so caching headers and data in
>    separate files gets very messy if you want to keep consistency.

Keep in mind that HTTP/1.1 compliance requires that the headers be
updatable without changing the body.

> * You can't use tempfiles since you want to be able to figure out
>    where the data is to be able to reply while caching.
> * You want to know the size of the data in order to tell when you're
>    done (ie the current size of a file isn't necessarily the real size
>    of the body since it might be caching while we're reading it).

The cache already wants to know the size of the data so that it can decide
whether it's prepared to try and cache the file in the first place, so in
theory this should not be a problem.

> In any case the patch is more or less finished, independent testing
> and auditing haven't been done yet but I can submit a preliminary
> jumbo-patch if people are interested in having a look at it now.

Post it, people can take a look.

Regards,
Graham
--