httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Querna <p...@querna.org>
Subject Re: mod_cache: disk layout for vary support
Date Mon, 11 Oct 2010 04:26:33 GMT
On Sun, Oct 10, 2010 at 8:56 AM, Graham Leggett <minfrin@sharp.fm> wrote:
> Hi all,
>
> One of the things that needs to be fixed with mod_cache is the support for
> caching varying responses. In the current cache, we store it as below, as an
> additional directory tree below the original URL's directory tree. This
> wastes lots of inodes, and is very expensive to write.
>
> /tmp/cacheroot/
> /tmp/cacheroot//1uq
> /tmp/cacheroot//1uq/W@D
> /tmp/cacheroot//1uq/W@D/Fok
> /tmp/cacheroot//1uq/W@D/Fok/HRU
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5/im1
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5/im1/RSz
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5/im1/RSz/fCK
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5/im1/RSz/fCK/YHquMmA.data
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header.vary/thJ/bK5/im1/RSz/fCK/YHquMmA.header
>
> What I have in mind is to move the varied content into the main tree, like
> this:
>
> /tmp/cacheroot/
> /tmp/cacheroot//1uq
> /tmp/cacheroot//1uq/W@D
> /tmp/cacheroot//1uq/W@D/Fok
> /tmp/cacheroot//1uq/W@D/Fok/HRU
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62
> /tmp/cacheroot//1uq/W@D/Fok/HRU/I62/QSJf2JA.header
> /tmp/cacheroot//thJ
> /tmp/cacheroot//thJ/bK5
> /tmp/cacheroot//thJ/bK5/im1
> /tmp/cacheroot//thJ/bK5/im1/RSz
> /tmp/cacheroot//thJ/bK5/im1/RSz/fCK
> /tmp/cacheroot//thJ/bK5/im1/RSz/fCK/YHquMmA.data
> /tmp/cacheroot//thJ/bK5/im1/RSz/fCK/YHquMmA.header
>
> We reuse the same directory structure in the process, and keep the original
> QSJf2JA.header file indicating that the URL is a varied URL.

The problem with the second layout is that it makes it near impossible
to clear out all variants of a URL easily.

I think there are far more general issues with waste of inodes on the
cache disk format, than anything to do with varres.

I would rather change the defaults to use only two letters and two
levels deep for the cache directories, and probally restrict the
character set even further to just [a-zA-Z].

I think a case should be made for not using sub-directories inside the
.varies folder. Instead just flatten it out and put all the variants
inside a flat directory, rather than distributing them over the cache
again.

Thanks,

Paul

Mime
View raw message