httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niklas Edmundsson <>
Subject Re: mod_disk_cache: making commit_entity() atomic
Date Mon, 20 Sep 2010 10:52:11 GMT
On Fri, 17 Sep 2010, Graham Leggett wrote:

> On 17 Sep 2010, at 1:41 PM, Niklas Edmundsson wrote:
>> I personally favor designs needs at most O_EXCL style write locking.
>> Having been bitten by various lock-related issues over the years I'm in 
>> favor of a explicit-lock-free design if it can be done cleanly and with 
>> good performance.
>> If going this route, I'd suggest to put the entire path to the data file in 
>> the header and not just a uniqifying string (to make it easier to split 
>> hashing of header and data in the future).
> The problem with this is that if you bake the location of the file into the 
> cache, you would never be able to move the cache around.

Just store the path relative to the cache root.

> Is there a benefit to keeping headers and bodies separate?

As we cache files from an nfs mount, we hash on device:inode as a 
simple method of reducing duplicates of files (say a dozen URL:s all 
resolving to the same DVD image). We see a huge benefit of being able 
to do this as we get a grotesque amount of data duplication otherwise.

So we usually have multiple header files all pointing to the same data 

For the more generic cache it might also be useful provided that you 
have a mechanism to identify duplicated data, the only thing I can 
think of is hashing on the data block but that isn't really feasible 
for large files. I suspect there might be cases where there exists 
usecases with a backend that can provide hints for this though.

  Niklas Edmundsson, Admin @ {acc,hpc2n}      |
  "Come on, higher now! A watcher scoffs at gravity!" - Giles

View raw message