httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "josh rotenberg" <joshrotenb...@gmail.com>
Subject Re: store_headers in memcache and diskcache
Date Sat, 09 Feb 2008 04:36:47 GMT
Dirk, my comments inline ...

On Feb 8, 2008 11:19 AM, Dirk-Willem van Gulik <dirkx@webweaving.org> wrote:
.. snip snip ..

> Ok - more or less merged a lot between both modules (2) --- al the
> good bits are yours :) Does that make sense to you ?

Hah! Then there is a lot of work to do!

> Issues I am particularly torn over are:
>
> -       change from re-alloc to brigades -- but this means we have 100-500k
> chunks sitting in memory 2x or 3x
>
>         ---> ultimately perhaps apr_memcache needs to understands the
> brigades perhaps.

I thought a lot about this, actually. An
apr_memcache_(set|get)_brigade or something would be handy. I was also
really wishing for the memcached 'append' command, which didn't exist
at the time. If we check the version on server init it could do one or
the other.

> -       passing of the info glob -- I guess we should make it perhaps number
> \0number\0 or something for convenience and speed ?

Yeah, agreed. I'm not particularly happy/bound to my serialization of
the info and headers, its not the most optimal solution. My goals at
the time were basically to make it work, and to some degree make it
human readable for testing.

> -       I am making MD5 keys -- you are trusting the URI. What is best
> really ?

Again, I was mainly trying to keep things simple for testing, so I
have no issue with a better alternative.


> -       create/open -- we could fetch it all already then.
> -       Or defer to the actual header/body routines - as you and I do now.
>
> -       Or just 'punt' and store the thing as one big wack - as generally
> you are
>         propably going to need both ?

I struggled with this one a lot. In some ways an all or nothing
approach seems better. Less setting and getting, its there or its not.
I guess I was mainly trying to follow the mod_disk_cache model, and in
some ways conform to what the provider api for caching seems to
assume, that the two are separate. I think I was also discussing with
someone the possibility of having body data auto-populate in the cache
(via something completely outside of apache) and it sounded cool to be
able to dump content into the cache without having to worry about
faking up a header entry. And/or the possibility of allowing headers
and bodies to be stored in different servers/pools, so that perhaps
header info is expiring or LRU'ing at a different rate. Just thoughts,
really, though, not sure if it makes sense to overdesign for it.

If you want to collaborate on the google project let me know and I can
add you, or you can take over, or however.

Thanks,

Josh

Mime
View raw message